-- Logs begin at Wed 2022-06-22 11:03:53 UTC, end at Wed 2022-06-22 12:15:11 UTC. -- Jun 22 11:30:52.090106 dv-cinder-836973-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 22 11:30:52.097276 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 22 11:30:52.102691 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jun 22 11:30:52 2022] *** Jun 22 11:30:52.102691 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 22 11:30:52.102691 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 22 11:30:52.102691 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: nodename: dv-cinder-836973-9-iscsi Jun 22 11:30:52.102691 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: machine: x86_64 Jun 22 11:30:52.102691 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: clock source: unix Jun 22 11:30:52.102691 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: pcre jit disabled Jun 22 11:30:52.102691 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: detected number of CPU cores: 4 Jun 22 11:30:52.102691 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: current working directory: / Jun 22 11:30:52.102691 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: detected binary path: /usr/bin/uwsgi-core Jun 22 11:30:52.102691 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: your processes number limit is 47680 Jun 22 11:30:52.102691 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: your memory page size is 4096 bytes Jun 22 11:30:52.102691 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: detected max file descriptor number: 2048 Jun 22 11:30:52.102691 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: lock engine: pthread robust mutexes Jun 22 11:30:52.102691 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: thunder lock: enabled Jun 22 11:30:52.102691 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 22 11:30:52.102691 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 22 11:30:52.147677 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: Python main interpreter initialized at 0x5620765e79c0 Jun 22 11:30:52.147677 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: python threads support enabled Jun 22 11:30:52.147677 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: your server socket listen backlog is limited to 100 connections Jun 22 11:30:52.147677 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: your mercy for graceful operations on workers is 90 seconds Jun 22 11:30:52.148057 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: mapped 403077 bytes (393 KB) for 2 cores Jun 22 11:30:52.148128 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: *** Operational MODE: preforking *** Jun 22 11:30:52.148397 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: *** uWSGI is running in multiple interpreter mode *** Jun 22 11:30:52.148397 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: spawned uWSGI master process (pid: 86757) Jun 22 11:30:52.148659 dv-cinder-836973-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 22 11:30:52.149486 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: spawned uWSGI worker 1 (pid: 86768, cores: 1) Jun 22 11:30:52.149590 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: spawned uWSGI worker 2 (pid: 86769, cores: 1) Jun 22 11:30:52.149774 dv-cinder-836973-9-iscsi devstack@keystone.service[86757]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 22 11:30:57.631623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 22 11:30:57.631623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: from cryptography.utils import int_from_bytes Jun 22 11:30:57.632409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 22 11:30:57.632409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: from cryptography.utils import int_from_bytes Jun 22 11:30:57.632542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 22 11:30:57.632542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: from cryptography.utils import int_from_bytes Jun 22 11:30:57.632542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 22 11:30:57.632542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: from cryptography.utils import int_from_bytes Jun 22 11:30:57.681641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 22 11:30:57.681814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 22 11:30:57.681892 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 22 11:30:57.681981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 22 11:30:57.682054 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] command line args: [] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 22 11:30:57.682135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] command line args: [] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 22 11:30:57.682276 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 22 11:30:57.682360 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 22 11:30:57.682462 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ================================================================================ {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 22 11:30:57.682540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ================================================================================ {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 22 11:30:57.682719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] admin_token = **** {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.682807 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] admin_token = **** {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.682970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] config_dir = [] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.683048 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] config_dir = [] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.683156 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.683239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.683364 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] config_source = [] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.683454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] config_source = [] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.683555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] debug = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.683630 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] debug = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.683754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.684064 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.684190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.684290 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.684390 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.684489 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.684590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.684690 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.684784 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.684878 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.685053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] list_limit = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.685172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] list_limit = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.685290 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] log_config_append = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.685392 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] log_config_append = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.685495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.685599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.685709 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] log_dir = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.685806 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] log_dir = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.685914 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] log_file = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.686012 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] log_file = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.686098 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.686223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.686301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.686422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.686507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.686586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.686659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.686733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.686806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.686936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.687054 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.687139 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.687219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.687337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.687445 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.687579 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.687664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.687737 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.687858 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.687935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.688031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.688121 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.688265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.688385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.688514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.688602 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.688763 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.688855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.689039 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.689117 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.689529 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.689655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.689741 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] publish_errors = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.689813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] publish_errors = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.689927 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.690013 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.690143 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.690221 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.690320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.690403 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.690566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.690660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.690801 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.690889 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.691031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] standard_threads = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.691155 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] standard_threads = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.691260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.691337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.691529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.691618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.691692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.691773 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.691871 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] use_journal = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.691950 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] use_journal = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.692027 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] use_json = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.692098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] use_json = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.692215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] use_stderr = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.692299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] use_stderr = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.692373 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] use_syslog = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.692449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] use_syslog = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.692585 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.692663 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 22 11:30:57.692859 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.692953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.692953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.693750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.693827 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.693904 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.694067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.694154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.694316 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.694393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.694553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.694636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.694802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.694891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.695039 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.695117 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.695286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.695392 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.695562 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.695652 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.695812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.695895 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.696046 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.696129 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.696292 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.696382 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.696527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.696609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.696773 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.696851 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.697019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.697179 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.697255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.697325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.697496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.697575 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.697750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.697838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.697991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.698073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.698224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.698300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.698474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.698566 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.698734 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.698811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.698984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.699060 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.699230 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.699316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.699484 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.699562 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.699714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.699800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.699943 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.700021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.700176 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.700260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.700423 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.700500 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.700667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.700744 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.700899 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.700976 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.701128 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.701210 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.701374 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.701452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.701621 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.701703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.701856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.701933 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.702103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.702181 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.702333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.702416 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.702568 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.702645 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.702791 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.702865 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.703019 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.703098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.703248 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.703323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.703500 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.703579 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.703731 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.703809 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.703966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.704048 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.704216 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.704301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.704446 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.704555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.704682 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.704792 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.704899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.704994 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.705132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.705238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.705368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.705472 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.705598 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.705680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.705833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.705916 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.706070 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.706187 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.706307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.706389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.706543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.706620 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.706776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.706853 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.707010 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.707091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.707238 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.707324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.707511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.707605 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.707748 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.707835 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.707985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.708072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.708218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.708305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.708477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.708556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.708714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.708796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.709519 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.709636 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.709762 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.709880 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.710002 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.710127 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.710276 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.710377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.710518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.710620 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.710775 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.710863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.711016 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.711095 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.711254 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.711341 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.711525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.711604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.711764 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.711842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.712011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.712105 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.712249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.712333 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.712486 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.712564 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.712727 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.712812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.712962 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.713039 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.713198 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.713280 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.713431 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.713516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.713668 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.713752 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.713900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.713977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.714161 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.714246 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.714401 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.714480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.714639 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.714716 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.714876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.715028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.715104 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.715175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.715341 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.715461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.715614 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.715704 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.715851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.715944 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.716082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.716168 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.716312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.716389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.716541 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.716629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.716772 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.716850 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.717001 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.717078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.717254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.717331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.717473 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.717555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.717720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.717797 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.717955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.718040 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.718210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.718287 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.718468 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.718551 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.718708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.718786 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.718938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] auth.external = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.719015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] auth.external = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.719168 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.719244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.719488 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.719610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.719726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.719807 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.719959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] auth.password = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.720038 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] auth.password = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.720198 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] auth.token = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.720277 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] auth.token = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.720441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.720522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.720675 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.720752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.720921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.721006 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.721152 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.721231 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.721399 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.721477 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.721645 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.721723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.721877 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.721956 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.722109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] credential.caching = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.722186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] credential.caching = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.722356 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.722433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.722604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.722683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.722852 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.722929 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.723086 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.723165 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.723328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.723450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.723584 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.723664 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.723848 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.723927 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.724102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.724182 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.724344 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.724441 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.724625 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.724704 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.724912 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.724998 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.725157 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.725236 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.725418 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.725496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.725660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.725739 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.725914 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.725992 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.726150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] federation.caching = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.726228 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] federation.caching = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.726389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.726468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.726635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.726713 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.726882 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.726960 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.727113 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.727192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.727378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.727459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.727609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.727698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.727866 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.727947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.728103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.728182 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.728358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.728444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.728593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.728671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.728857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.728939 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.729074 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] identity.caching = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.729223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] identity.caching = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.729324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.729404 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.729569 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.729646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.729812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.729906 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.730037 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.730116 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.730284 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.730363 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.730512 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.730598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.730748 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.730827 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.730995 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.731074 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.731247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.731333 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.731507 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.731589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.731738 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.731816 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.731969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.732055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.732212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.732291 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.732458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.732536 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.732704 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.732789 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.732960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.733038 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.733206 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.733280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.733478 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.733565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.733716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.733795 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.733953 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.734031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.734186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.734273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.734423 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.734501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.734654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.734732 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.734888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.734974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.735127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.735205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.735395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.735479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.735635 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.735727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.735866 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.735944 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.736115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.736200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.736362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.736451 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.736595 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.736673 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.736841 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.736918 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.737083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.737168 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.737313 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.737390 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.737547 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.737625 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.737776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.737865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.738012 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.738090 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.738251 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.738327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.738490 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.738576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.738727 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.738805 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.738964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.739041 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.739214 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.739301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.739481 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.739560 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.739715 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.739794 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.739946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.740036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.740218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.740297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.740466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.740544 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.740699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.740778 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.740933 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.741011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.741165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.741242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.741395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.741474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.741633 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.741710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.741891 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.741971 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.742122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.742201 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.742378 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.742468 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.742615 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.742693 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.742859 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.742929 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.743092 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.743160 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.743405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.743488 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.743722 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.743792 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.743853 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.743912 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.744023 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.744105 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.744254 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.744335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.744500 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.744577 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.744744 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.744822 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.744988 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.745068 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.745231 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.745309 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.745472 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.745550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.745701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.745786 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.745946 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.746024 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.746182 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.746259 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.746417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.746506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.746653 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.746731 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.746887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.746965 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.747134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.747213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.747405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.747489 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.747645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.747722 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.747899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.747980 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.748128 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.748207 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.748371 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.748448 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.748608 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.748688 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.748842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.748919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.749077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.749155 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.749326 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.749404 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.749577 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.749660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.749800 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.749878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.750033 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.750112 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.750273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] resource.caching = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.750357 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] resource.caching = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.750521 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.750599 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.750768 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.750848 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.751003 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.751082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.751252 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.751330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.751536 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.751616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.751772 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.751851 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.752019 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.752108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.752258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.752334 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.752484 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.752563 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.752718 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] role.caching = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.752797 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] role.caching = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.752964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] role.driver = sql {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.753044 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] role.driver = sql {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.753192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.753270 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.753440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.753517 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.753688 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.753767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.753935 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.754012 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.754180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.754266 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.754426 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.754505 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.754671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.754749 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.754915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.755002 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.755163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.755241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.755423 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.755530 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.756185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.756345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.756451 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.756787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.756943 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.757069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.757219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.757362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.757713 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.757950 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.758092 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.758224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.758351 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.758498 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.758677 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.758811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.758942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.759154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.759292 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.761025 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.761123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.761202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.761277 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.761398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.761524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.761666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.761803 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.761931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.761931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.762162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.762306 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.762306 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.762549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.762662 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.762777 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.762910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.763053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.763185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.763313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.763539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.763622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.763696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] token.caching = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.763784 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.763862 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.763862 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.763992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.764064 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.764064 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.764201 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.764528 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.764528 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.764698 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.764832 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.764968 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.765113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.765113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.765359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] token.caching = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.765496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.765632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.765760 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.765857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.765933 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.766006 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.766078 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.766078 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.766215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.766297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.766297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.766429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.766500 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.766607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.766688 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.connection = **** {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.766924 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.767051 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.767175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.767478 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.772892 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.772892 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.773042 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.773115 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.773192 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.773266 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.773338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.773410 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.773482 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.773563 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.773637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.773709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.773781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.773853 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.773914 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.773983 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.774045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.774118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.774247 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.774340 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.774415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.connection = **** {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.774491 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.774574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.774658 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.774733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.774806 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.774875 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.774935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.774999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.775068 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.775068 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 22 11:30:57.775190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.775190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.775190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.775190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.775190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.775190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.775190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.775190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.775190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.775747 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.775747 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.775747 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.775747 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.775747 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 22 11:30:57.775747 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86768) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 22 11:30:57.803314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:57.803556 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:57.803781 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:57.805682 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:57.805911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:57.806133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:57.812349 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:57.814646 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:57.839900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:57.840118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:57.840250 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:57.840483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:57.983733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:57.984279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.033840 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.034115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.035146 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.035436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.098230 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.098457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.098640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.098795 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.098942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.099072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.099200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.099325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.099484 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.099614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.099614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.099855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.124474 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.124704 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.124834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.124975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.125099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.125222 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.125355 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.125479 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.125603 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.125727 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.125849 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.125970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.126101 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.126101 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.137238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.137502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.150134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.150368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.150498 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.150623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.150746 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.150869 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.151002 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.151125 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.151259 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.151414 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.151539 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.151665 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.151802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.151802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86768) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 22 11:30:58.168715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.168909 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.169048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 22 11:30:58.169196 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 22 11:30:58.169325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.169463 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.169588 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 22 11:30:58.169713 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 22 11:30:58.169846 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.169970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.170091 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.170214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.170338 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.170459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.170590 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.170714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.181786 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.181942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.182068 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.182197 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.182335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.182459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.182581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.182708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.190872 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.191016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.194806 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.194959 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.195084 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.195206 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.195338 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.195487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.195614 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.195740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.195867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.196005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.196132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.196257 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.196392 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.196519 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.196644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.196779 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.196903 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.197030 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.197156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.197281 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.197413 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.197540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.223801 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.223963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.225442 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.225585 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.225711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.225832 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.230985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.231124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.231383 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.231564 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.231690 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.231825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.238938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.239103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.239243 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.239387 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.244249 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.244390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.244515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.244650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.244868 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.245015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.245151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.245276 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.245399 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.245529 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.245651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.245773 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.245894 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.246015 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.246143 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.246268 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.265127 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.265287 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.265413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.265549 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.270047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.270176 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.273736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.273870 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.274133 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.274257 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.279190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.279337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.282826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.282965 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.286504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.286636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.286802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.287487 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.287615 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.287748 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.287902 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.287986 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.288180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.288250 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.288456 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.288526 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.308230 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.308455 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.308584 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.308725 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.308854 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.308995 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.309123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.309251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.317014 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.317165 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.318736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.318879 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.319003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.319213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.322329 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.322510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.322716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.322912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.323104 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.323309 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.323467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.323595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.323719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.323845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.336890 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.337130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.337304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.337447 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.337585 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.337728 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.337863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.337996 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.339499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.339650 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.339794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.339924 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.340053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.340192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.358629 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.358873 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.361979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.362128 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.365911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.366063 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.367689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.367834 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.369331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.369492 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.369623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.369750 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.369888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.370018 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.377239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.377420 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.378856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.379018 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.382177 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.382328 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.382471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.382602 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.382848 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.382993 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.383126 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.383257 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.392675 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.392854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.393005 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.393134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.393262 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.393402 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.401219 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.401471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86768) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 22 11:30:58.401471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.401729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.401868 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.401998 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.402126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.402282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.402449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.402637 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.402851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.403050 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.403269 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.403443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.403443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.403702 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.403943 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.404081 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.404212 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.404339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.404487 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.404693 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.404831 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.404965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.405093 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.405219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.405345 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.405479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86768) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 22 11:30:58.443714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5620765e79c0 pid: 86769 (default app) Jun 22 11:30:58.443899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5620765e79c0 pid: 86768 (default app) Jun 22 11:30:58.447400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e160e-5340-480f-a54a-522f716f3dc8 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:30:58.447648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e160e-5340-480f-a54a-522f716f3dc8 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:30:58.447933 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e160e-5340-480f-a54a-522f716f3dc8 None None] PATH_INFO: `/v3` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:30:58.452161 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 791 bytes} [Wed Jun 22 11:30:58 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:06.246238 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac37fde-d76e-458e-80f2-c4169179c689 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:06.246773 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac37fde-d76e-458e-80f2-c4169179c689 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:06.246773 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac37fde-d76e-458e-80f2-c4169179c689 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:06.251073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1/2] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:06.257187 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36a64e-9f8f-483b-bbef-c212317e8431 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:06.257608 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36a64e-9f8f-483b-bbef-c212317e8431 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:06.257735 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36a64e-9f8f-483b-bbef-c212317e8431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:06.325522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: INFO dbcounter [None req-1b36a64e-9f8f-483b-bbef-c212317e8431 None None] Registered counter for database keystone Jun 22 11:31:06.347683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG oslo_db.sqlalchemy.engines [None req-1b36a64e-9f8f-483b-bbef-c212317e8431 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86769) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 22 11:31:06.354528 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writer thread running {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 22 11:31:06.531208 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG passlib.handlers.bcrypt [None req-1b36a64e-9f8f-483b-bbef-c212317e8431 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86769) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 22 11:31:06.531556 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG passlib.handlers.bcrypt [None req-1b36a64e-9f8f-483b-bbef-c212317e8431 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86769) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 22 11:31:06.586399 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-1b36a64e-9f8f-483b-bbef-c212317e8431 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:06.588224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-1b36a64e-9f8f-483b-bbef-c212317e8431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:06.711791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 2/3] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 22 11:31:06.855978 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac37fde-d76e-458e-80f2-c4169179c689 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:06.926866 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: INFO dbcounter [None req-1ac37fde-d76e-458e-80f2-c4169179c689 None None] Registered counter for database keystone Jun 22 11:31:06.947932 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG oslo_db.sqlalchemy.engines [None req-1ac37fde-d76e-458e-80f2-c4169179c689 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86768) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 22 11:31:06.954252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writer thread running {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 22 11:31:07.074419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac37fde-d76e-458e-80f2-c4169179c689 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:07.078613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8c321-a66e-4700-a70d-32dcc6ec4f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:07.079276 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8c321-a66e-4700-a70d-32dcc6ec4f10 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:07.079580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8c321-a66e-4700-a70d-32dcc6ec4f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:07.079853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8c321-a66e-4700-a70d-32dcc6ec4f10 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:07.088542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8c321-a66e-4700-a70d-32dcc6ec4f10 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:07.139795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d9c8c321-a66e-4700-a70d-32dcc6ec4f10 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 22 11:31:07.140136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8c321-a66e-4700-a70d-32dcc6ec4f10 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:07.150608 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 2/4] 10.222.0.36 () {58 vars in 1324 bytes} [Wed Jun 22 11:31:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 22 11:31:07.156516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36a64e-9f8f-483b-bbef-c212317e8431 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:07.171596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36a64e-9f8f-483b-bbef-c212317e8431 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:07.177506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77fad9-2f08-47d1-be10-62b1b4c57b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:07.178625 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77fad9-2f08-47d1-be10-62b1b4c57b9d admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:07.178912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77fad9-2f08-47d1-be10-62b1b4c57b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:07.179187 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77fad9-2f08-47d1-be10-62b1b4c57b9d admin admin] PATH_INFO: `/v3/projects/c77776cd83344a69a0a4fbb7efadd00f` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:07.183892 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77fad9-2f08-47d1-be10-62b1b4c57b9d admin admin] RBAC: Authorizing `identity:get_project(project_id=c77776cd83344a69a0a4fbb7efadd00f)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:07.238038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7c77fad9-2f08-47d1-be10-62b1b4c57b9d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 22 11:31:07.238389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77fad9-2f08-47d1-be10-62b1b4c57b9d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:07.243138 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 3/5] 10.222.0.36 () {56 vars in 1207 bytes} [Wed Jun 22 11:31:07 2022] GET /identity/v3/projects/c77776cd83344a69a0a4fbb7efadd00f => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:31:09.268669 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3da8e2-6c88-4fc1-9701-3c304628909f None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:09.269168 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3da8e2-6c88-4fc1-9701-3c304628909f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:09.269168 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3da8e2-6c88-4fc1-9701-3c304628909f None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:09.274000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 3/6] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:31:09.280156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc7525-9357-4beb-9c3d-7097dd3f309f None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:09.280629 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc7525-9357-4beb-9c3d-7097dd3f309f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:09.280721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc7525-9357-4beb-9c3d-7097dd3f309f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:09.300097 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-16fc7525-9357-4beb-9c3d-7097dd3f309f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:09.301636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-16fc7525-9357-4beb-9c3d-7097dd3f309f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:09.309500 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 4/7] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 22 11:31:09.332064 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3da8e2-6c88-4fc1-9701-3c304628909f None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:09.344798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3da8e2-6c88-4fc1-9701-3c304628909f None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:09.349035 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a83e1-e4b7-4bb1-ba8e-a2b84afb66f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:09.350319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a83e1-e4b7-4bb1-ba8e-a2b84afb66f9 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:09.350596 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a83e1-e4b7-4bb1-ba8e-a2b84afb66f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:09.350872 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a83e1-e4b7-4bb1-ba8e-a2b84afb66f9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:09.359525 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a83e1-e4b7-4bb1-ba8e-a2b84afb66f9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:09.361336 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a83e1-e4b7-4bb1-ba8e-a2b84afb66f9 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:09.370958 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 4/8] 10.222.0.36 () {58 vars in 1324 bytes} [Wed Jun 22 11:31:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 22 11:31:09.377349 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc7525-9357-4beb-9c3d-7097dd3f309f None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:09.388821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc7525-9357-4beb-9c3d-7097dd3f309f None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:09.393761 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff00c7-f39c-47f3-923d-ae5bd01633d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:09.394639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff00c7-f39c-47f3-923d-ae5bd01633d2 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:09.394922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff00c7-f39c-47f3-923d-ae5bd01633d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:09.395200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff00c7-f39c-47f3-923d-ae5bd01633d2 admin admin] PATH_INFO: `/v3/users/8441761690594b9d8a423147d5d6d62d` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:09.397218 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff00c7-f39c-47f3-923d-ae5bd01633d2 admin admin] RBAC: Authorizing `identity:get_user(user_id=8441761690594b9d8a423147d5d6d62d)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:09.399085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff00c7-f39c-47f3-923d-ae5bd01633d2 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:09.400834 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 5/9] 10.222.0.36 () {56 vars in 1198 bytes} [Wed Jun 22 11:31:09 2022] GET /identity/v3/users/8441761690594b9d8a423147d5d6d62d => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:11.616118 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711adc91-5126-4f84-9f4b-79bfeb9748e5 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:11.616653 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711adc91-5126-4f84-9f4b-79bfeb9748e5 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:11.616653 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711adc91-5126-4f84-9f4b-79bfeb9748e5 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:11.618042 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 5/10] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:11.623556 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6803e-42c6-4724-8402-b8a19d1a498b None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:11.624007 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6803e-42c6-4724-8402-b8a19d1a498b None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:11.624128 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6803e-42c6-4724-8402-b8a19d1a498b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:11.641867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d2a6803e-42c6-4724-8402-b8a19d1a498b None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:11.643059 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d2a6803e-42c6-4724-8402-b8a19d1a498b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:11.653116 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffbb8c-27a3-4d4a-96f4-428942cce486 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:11.653443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffbb8c-27a3-4d4a-96f4-428942cce486 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:11.653731 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffbb8c-27a3-4d4a-96f4-428942cce486 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:11.655243 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 6/11] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:31:11.660428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2d100-2e4f-4cc7-8783-c99ba7f525c1 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:11.660829 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2d100-2e4f-4cc7-8783-c99ba7f525c1 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:11.661111 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2d100-2e4f-4cc7-8783-c99ba7f525c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:11.702723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG passlib.handlers.bcrypt [None req-6af2d100-2e4f-4cc7-8783-c99ba7f525c1 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86768) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 22 11:31:11.703244 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG passlib.handlers.bcrypt [None req-6af2d100-2e4f-4cc7-8783-c99ba7f525c1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86768) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 22 11:31:11.730357 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 6/12] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:11.749377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-6af2d100-2e4f-4cc7-8783-c99ba7f525c1 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:11.751025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-6af2d100-2e4f-4cc7-8783-c99ba7f525c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:11.754504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6803e-42c6-4724-8402-b8a19d1a498b None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:11.780580 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:11.857606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 7/13] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 22 11:31:11.873092 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6803e-42c6-4724-8402-b8a19d1a498b None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:11.881110 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2d100-2e4f-4cc7-8783-c99ba7f525c1 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:11.920570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688321f-75a9-47ff-8c7c-e66f15375472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:11.921379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688321f-75a9-47ff-8c7c-e66f15375472 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:11.921665 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688321f-75a9-47ff-8c7c-e66f15375472 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:11.921946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688321f-75a9-47ff-8c7c-e66f15375472 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:11.937325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0688321f-75a9-47ff-8c7c-e66f15375472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:11.949151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-0688321f-75a9-47ff-8c7c-e66f15375472 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:31:11.949151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:11.949151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:11.949151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:11.949151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:11.949151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:11.949151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:11.949151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:11.949151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:11.949151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:11.949151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:11.949151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:11.949151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:11.949151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:11.949151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:11.949151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:11.950283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:11.950283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:11.950283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:11.950283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:11.950283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:11.950283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:11.950283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:11.950283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:11.950283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:11.950283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:11.950283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:11.950283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:11.950283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:11.950283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:11.950283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:11.950283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:11.951170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:11.951170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:11.951170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:11.951170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:11.951170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:11.951170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:11.951170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:11.951170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:11.951170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:11.951170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:11.951170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:31:11.951170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:11.951170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 7/14] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:31:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:31:11.956519 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688321f-75a9-47ff-8c7c-e66f15375472 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:12.010848 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2d100-2e4f-4cc7-8783-c99ba7f525c1 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:12.013343 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:12.047920 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:12.054046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a10e5-517d-411c-9043-ea09273e853b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:12.054987 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a10e5-517d-411c-9043-ea09273e853b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:12.055291 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a10e5-517d-411c-9043-ea09273e853b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:12.055600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a10e5-517d-411c-9043-ea09273e853b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:12.067701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688321f-75a9-47ff-8c7c-e66f15375472 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:12.105008 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffb8bf-8569-44aa-8722-529e93ded1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:12.105710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffb8bf-8569-44aa-8722-529e93ded1bd None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:12.105995 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffb8bf-8569-44aa-8722-529e93ded1bd None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:12.106275 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffb8bf-8569-44aa-8722-529e93ded1bd None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:12.107082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffb8bf-8569-44aa-8722-529e93ded1bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:12.107419 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffb8bf-8569-44aa-8722-529e93ded1bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:12.108881 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffb8bf-8569-44aa-8722-529e93ded1bd None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:12.122041 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 8/15] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:12.129200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffb8bf-8569-44aa-8722-529e93ded1bd None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:12.130128 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a10e5-517d-411c-9043-ea09273e853b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:12.131713 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a10e5-517d-411c-9043-ea09273e853b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:12.241217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffb8bf-8569-44aa-8722-529e93ded1bd None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:12.278333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69841d-993f-43a1-adae-5c623451ff76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:12.279217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69841d-993f-43a1-adae-5c623451ff76 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:12.279532 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69841d-993f-43a1-adae-5c623451ff76 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:12.279812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69841d-993f-43a1-adae-5c623451ff76 None admin] PATH_INFO: `/v3/users/8441761690594b9d8a423147d5d6d62d` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:12.280592 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 8/16] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:31:12.281070 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69841d-993f-43a1-adae-5c623451ff76 None admin] RBAC: Authorizing `identity:get_user(user_id=8441761690594b9d8a423147d5d6d62d)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:12.282546 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69841d-993f-43a1-adae-5c623451ff76 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:12.284408 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 9/17] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:12 2022] GET /identity/v3/users/8441761690594b9d8a423147d5d6d62d => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:31:12.286492 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a10e5-517d-411c-9043-ea09273e853b None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:12.290241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:12.290563 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69841d-993f-43a1-adae-5c623451ff76 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:12.305982 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:12.389791 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a10e5-517d-411c-9043-ea09273e853b None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:12.394981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69841d-993f-43a1-adae-5c623451ff76 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:12.426950 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec0826-99ab-4fec-b0db-da2b8a473657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:12.427813 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec0826-99ab-4fec-b0db-da2b8a473657 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:12.428092 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec0826-99ab-4fec-b0db-da2b8a473657 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:12.428366 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec0826-99ab-4fec-b0db-da2b8a473657 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:12.433518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3295fa2-9a0c-4f32-ad1d-009686ef823b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:12.434323 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3295fa2-9a0c-4f32-ad1d-009686ef823b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:12.434612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3295fa2-9a0c-4f32-ad1d-009686ef823b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:12.435010 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3295fa2-9a0c-4f32-ad1d-009686ef823b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:12.436129 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3295fa2-9a0c-4f32-ad1d-009686ef823b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:12.437601 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3295fa2-9a0c-4f32-ad1d-009686ef823b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:12.439243 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 10/18] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:31:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:12.445103 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3295fa2-9a0c-4f32-ad1d-009686ef823b None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:12.447859 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec0826-99ab-4fec-b0db-da2b8a473657 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:12.449438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec0826-99ab-4fec-b0db-da2b8a473657 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:12.470833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-b5ec0826-99ab-4fec-b0db-da2b8a473657 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:31:12.470833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:12.470833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:12.470833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:12.470833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:12.470833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:12.470833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:12.470833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:12.470833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:12.470833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:12.470833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:12.470833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:12.470833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:31:12.470833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:31:12.470833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:31:12.471769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:31:12.471769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:12.471769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:12.471769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:12.471769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:12.471769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:12.471769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:12.471769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:12.471769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:12.471769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:12.471769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:12.471769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:12.471769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:12.471769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:12.471769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:12.471769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:12.471769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:12.472596 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:31:12.472596 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:31:12.472596 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:31:12.472596 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:31:12.472596 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 9/19] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:31:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:12.478508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec0826-99ab-4fec-b0db-da2b8a473657 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:12.532701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:12.545324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:12.552670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3295fa2-9a0c-4f32-ad1d-009686ef823b None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:12.589893 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8fdd1-b843-4699-8240-39b7c7e61a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:12.590255 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8fdd1-b843-4699-8240-39b7c7e61a60 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:12.590539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8fdd1-b843-4699-8240-39b7c7e61a60 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:12.590817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8fdd1-b843-4699-8240-39b7c7e61a60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:12.591156 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec0826-99ab-4fec-b0db-da2b8a473657 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:12.591697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8fdd1-b843-4699-8240-39b7c7e61a60 None admin] RBAC: Adding query filter params (role.id=96c584428d924fdba9671949f760c406, scope.domain.id=default, user.id=8441761690594b9d8a423147d5d6d62d) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:12.592009 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8fdd1-b843-4699-8240-39b7c7e61a60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:12.593461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8fdd1-b843-4699-8240-39b7c7e61a60 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:12.600635 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 11/20] 10.222.0.36 () {56 vars in 1322 bytes} [Wed Jun 22 11:31:12 2022] GET /identity/v3/role_assignments?user.id=8441761690594b9d8a423147d5d6d62d&scope.domain.id=default&role.id=96c584428d924fdba9671949f760c406 => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:31:12.627086 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b03a9-8a86-4657-b569-add725fec0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:12.627807 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b03a9-8a86-4657-b569-add725fec0c2 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:12.628091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b03a9-8a86-4657-b569-add725fec0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:12.628371 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b03a9-8a86-4657-b569-add725fec0c2 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:12.629124 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b03a9-8a86-4657-b569-add725fec0c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:12.629437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b03a9-8a86-4657-b569-add725fec0c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:12.631494 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b03a9-8a86-4657-b569-add725fec0c2 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:12.652013 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 10/21] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:31:15.259840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc0ff6-db56-4efb-8b72-a05a94ef6486 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:15.260398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc0ff6-db56-4efb-8b72-a05a94ef6486 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:15.260398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc0ff6-db56-4efb-8b72-a05a94ef6486 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:15.261686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 12/22] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:15.265608 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571aa4e-9e43-4f12-9a84-30a6139851a1 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:15.266043 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571aa4e-9e43-4f12-9a84-30a6139851a1 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:15.266390 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571aa4e-9e43-4f12-9a84-30a6139851a1 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:15.267700 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0d235-9f30-4c1e-acdc-faf0baf17845 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:15.267808 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 11/23] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:15.268265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0d235-9f30-4c1e-acdc-faf0baf17845 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:15.268548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0d235-9f30-4c1e-acdc-faf0baf17845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:15.273489 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b32170-4f10-485e-aee0-24ca2bfdab64 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:15.273777 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b32170-4f10-485e-aee0-24ca2bfdab64 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:15.274060 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b32170-4f10-485e-aee0-24ca2bfdab64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:15.291994 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-26b0d235-9f30-4c1e-acdc-faf0baf17845 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:15.293193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-26b0d235-9f30-4c1e-acdc-faf0baf17845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:15.300369 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-35b32170-4f10-485e-aee0-24ca2bfdab64 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:15.301588 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-35b32170-4f10-485e-aee0-24ca2bfdab64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:15.419984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 12/24] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 22 11:31:15.449707 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 13/25] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 22 11:31:15.456128 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b32170-4f10-485e-aee0-24ca2bfdab64 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:15.488406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b0d235-9f30-4c1e-acdc-faf0baf17845 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:15.569524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:15.620868 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:15.642145 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b32170-4f10-485e-aee0-24ca2bfdab64 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:15.652937 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b0d235-9f30-4c1e-acdc-faf0baf17845 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:15.681217 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a23e4-df36-4270-81c5-7aa5c582aec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:15.682016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a23e4-df36-4270-81c5-7aa5c582aec3 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:15.682298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a23e4-df36-4270-81c5-7aa5c582aec3 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:15.682572 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a23e4-df36-4270-81c5-7aa5c582aec3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:15.694540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a23e4-df36-4270-81c5-7aa5c582aec3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:15.696072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a23e4-df36-4270-81c5-7aa5c582aec3 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:15.699774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f96fa-fbd9-45f0-b2bc-7fc636ee0490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:15.700702 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f96fa-fbd9-45f0-b2bc-7fc636ee0490 None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:15.701191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f96fa-fbd9-45f0-b2bc-7fc636ee0490 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:15.701678 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f96fa-fbd9-45f0-b2bc-7fc636ee0490 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:15.702640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f96fa-fbd9-45f0-b2bc-7fc636ee0490 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:15.704864 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f96fa-fbd9-45f0-b2bc-7fc636ee0490 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:15.717100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-364a23e4-df36-4270-81c5-7aa5c582aec3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:31:15.717100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:15.717100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:15.717100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:15.717100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:15.717100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:15.717100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:15.717100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:15.717100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:15.717100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:15.717100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:15.717100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:15.717100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:31:15.717100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:31:15.717100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:31:15.718815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:31:15.718815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:15.718815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:15.718815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:15.718815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:15.718815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:15.718815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:15.718815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:15.718815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:15.718815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:15.718815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:15.718815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:15.718815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:15.718815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:15.718815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:15.718815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:15.718815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:15.719911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:31:15.719911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:31:15.719911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:31:15.719911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:31:15.719911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 13/26] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:31:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:15.719911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232cd17-8bc5-4bad-9b1d-700d9cd69212 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:15.720478 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232cd17-8bc5-4bad-9b1d-700d9cd69212 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:15.720478 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232cd17-8bc5-4bad-9b1d-700d9cd69212 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:15.727896 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 14/27] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:15.727896 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e225144-f671-4484-aff2-89d7480ccdb8 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:15.727896 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e225144-f671-4484-aff2-89d7480ccdb8 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:15.727896 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e225144-f671-4484-aff2-89d7480ccdb8 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:15.729346 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 15/28] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:15.731248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b6721-b4d9-4025-a46c-20edb2632f14 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:15.731558 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b6721-b4d9-4025-a46c-20edb2632f14 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:15.731837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b6721-b4d9-4025-a46c-20edb2632f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:15.750171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-e26b6721-b4d9-4025-a46c-20edb2632f14 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:15.751482 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e26b6721-b4d9-4025-a46c-20edb2632f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:15.829191 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 16/29] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:15.830790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bc9d1-09b6-4fdb-a050-969330d8bcb5 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:15.831304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bc9d1-09b6-4fdb-a050-969330d8bcb5 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:15.831811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133bc9d1-09b6-4fdb-a050-969330d8bcb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:15.848882 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-133bc9d1-09b6-4fdb-a050-969330d8bcb5 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:15.850356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-133bc9d1-09b6-4fdb-a050-969330d8bcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:15.874916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:15.890523 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 14/30] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jun 22 11:31:15 2022] POST /identity/v3/roles => generated 219 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:31:15.896612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f96fa-fbd9-45f0-b2bc-7fc636ee0490 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:15.935128 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 17/31] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:15.936543 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bc9d1-09b6-4fdb-a050-969330d8bcb5 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:16.086133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133bc9d1-09b6-4fdb-a050-969330d8bcb5 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:16.128815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663a001-da69-4864-a34d-5bf5c319bdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:16.130169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663a001-da69-4864-a34d-5bf5c319bdc5 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:16.130698 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663a001-da69-4864-a34d-5bf5c319bdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:16.131186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663a001-da69-4864-a34d-5bf5c319bdc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:16.146541 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a663a001-da69-4864-a34d-5bf5c319bdc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:16.149796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f96fa-fbd9-45f0-b2bc-7fc636ee0490 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:16.156640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-a663a001-da69-4864-a34d-5bf5c319bdc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:31:16.156640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:16.156640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:16.156640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:16.156640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:16.156640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:16.156640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:16.156640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:16.156640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:16.156640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:16.156640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:16.156640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:16.156640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:16.156640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:16.156640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:16.156640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:16.157532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:16.157532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:16.157532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:16.157532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:16.157532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:16.157532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:16.157532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:16.157532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:16.157532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:16.157532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:16.157532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:16.157532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:16.157532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:16.157532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:16.157532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:16.157532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:16.158740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:16.158740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:16.158740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:16.158740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:16.158740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:16.158740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:16.158740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:16.158740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:16.158740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:16.158740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:16.158740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:31:16.158740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:31:16.158740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 18/32] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:31:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:16.158740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663a001-da69-4864-a34d-5bf5c319bdc5 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:16.181315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:16.226764 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbaf6a0-46e6-4488-a99e-b065a9e42795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:16.228034 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbaf6a0-46e6-4488-a99e-b065a9e42795 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:16.228446 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbaf6a0-46e6-4488-a99e-b065a9e42795 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:16.228737 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbaf6a0-46e6-4488-a99e-b065a9e42795 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:16.229752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbaf6a0-46e6-4488-a99e-b065a9e42795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:16.232712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbaf6a0-46e6-4488-a99e-b065a9e42795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:16.234612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbaf6a0-46e6-4488-a99e-b065a9e42795 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:16.236857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:16.264952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 15/33] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:16.265846 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbaf6a0-46e6-4488-a99e-b065a9e42795 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:16.293014 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663a001-da69-4864-a34d-5bf5c319bdc5 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:16.337828 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7ca71-7085-4ef0-bb64-8bab55a593fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:16.338431 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7ca71-7085-4ef0-bb64-8bab55a593fa None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:16.338880 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7ca71-7085-4ef0-bb64-8bab55a593fa None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:16.339167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7ca71-7085-4ef0-bb64-8bab55a593fa None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:16.340019 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7ca71-7085-4ef0-bb64-8bab55a593fa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:16.341525 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7ca71-7085-4ef0-bb64-8bab55a593fa None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:16.389984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbaf6a0-46e6-4488-a99e-b065a9e42795 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:16.425191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1bc85-7de6-44ec-9631-371667ffa7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:16.425867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1bc85-7de6-44ec-9631-371667ffa7fc None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:16.426157 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1bc85-7de6-44ec-9631-371667ffa7fc None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:16.426441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1bc85-7de6-44ec-9631-371667ffa7fc None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:16.427202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1bc85-7de6-44ec-9631-371667ffa7fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:16.428109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1bc85-7de6-44ec-9631-371667ffa7fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:16.429506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1bc85-7de6-44ec-9631-371667ffa7fc None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:16.438127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 16/34] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:16.438922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1bc85-7de6-44ec-9631-371667ffa7fc None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:16.479630 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 19/35] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jun 22 11:31:16 2022] POST /identity/v3/roles => generated 225 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:31:16.480784 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7ca71-7085-4ef0-bb64-8bab55a593fa None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:16.501628 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:16.577656 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1bc85-7de6-44ec-9631-371667ffa7fc None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:16.620833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83484c35-2c75-45cf-a592-5920931dc571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:16.621015 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83484c35-2c75-45cf-a592-5920931dc571 None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:16.621264 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83484c35-2c75-45cf-a592-5920931dc571 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:16.621515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83484c35-2c75-45cf-a592-5920931dc571 None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:16.622281 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83484c35-2c75-45cf-a592-5920931dc571 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:16.624011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83484c35-2c75-45cf-a592-5920931dc571 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:16.636658 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:16.662505 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 17/36] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jun 22 11:31:16 2022] POST /identity/v3/projects => generated 301 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:31:16.664336 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41647b4f-7acb-4ffb-b5e6-1e62871db241 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:16.664812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41647b4f-7acb-4ffb-b5e6-1e62871db241 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:16.665234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41647b4f-7acb-4ffb-b5e6-1e62871db241 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:16.666921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 18/37] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:16.668574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65408c48-bb4c-4b1e-b309-dba4d18be811 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:16.668878 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65408c48-bb4c-4b1e-b309-dba4d18be811 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:16.669159 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65408c48-bb4c-4b1e-b309-dba4d18be811 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:16.670782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 19/38] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:16.672367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7ca71-7085-4ef0-bb64-8bab55a593fa None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:16.676261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edefc9-af3f-44e0-bb45-6d967635f9b4 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:16.676602 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edefc9-af3f-44e0-bb45-6d967635f9b4 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:16.676602 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edefc9-af3f-44e0-bb45-6d967635f9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:16.695315 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-06edefc9-af3f-44e0-bb45-6d967635f9b4 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:16.696992 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-06edefc9-af3f-44e0-bb45-6d967635f9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:16.722767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ba165-245f-4449-b57d-8067852e2bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:16.723725 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ba165-245f-4449-b57d-8067852e2bcd None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:16.724014 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ba165-245f-4449-b57d-8067852e2bcd None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:16.724865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ba165-245f-4449-b57d-8067852e2bcd None admin] PATH_INFO: `/v3/users/8441761690594b9d8a423147d5d6d62d` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:16.726877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ba165-245f-4449-b57d-8067852e2bcd None admin] RBAC: Authorizing `identity:get_user(user_id=8441761690594b9d8a423147d5d6d62d)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:16.728641 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ba165-245f-4449-b57d-8067852e2bcd None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:16.730493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 20/39] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:16 2022] GET /identity/v3/users/8441761690594b9d8a423147d5d6d62d => generated 247 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:16.731938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f811bf-1f6e-4727-b398-2e25680dd66e None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:16.732234 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f811bf-1f6e-4727-b398-2e25680dd66e None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:16.732516 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f811bf-1f6e-4727-b398-2e25680dd66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:16.751067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-d4f811bf-1f6e-4727-b398-2e25680dd66e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:16.752895 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d4f811bf-1f6e-4727-b398-2e25680dd66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:16.807197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 20/40] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:16.815569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edefc9-af3f-44e0-bb45-6d967635f9b4 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:16.853365 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 21/41] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:16.853561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f811bf-1f6e-4727-b398-2e25680dd66e None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:16.858308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:16.930538 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edefc9-af3f-44e0-bb45-6d967635f9b4 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:16.941979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:16.966197 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f811bf-1f6e-4727-b398-2e25680dd66e None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:16.967272 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4365c6-cf05-4797-b805-9251e1ebf413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:16.968096 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4365c6-cf05-4797-b805-9251e1ebf413 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:16.968387 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4365c6-cf05-4797-b805-9251e1ebf413 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:16.968669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4365c6-cf05-4797-b805-9251e1ebf413 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:16.969872 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4365c6-cf05-4797-b805-9251e1ebf413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:16.971336 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4365c6-cf05-4797-b805-9251e1ebf413 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:16.972991 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 21/42] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:31:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:16.973729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4365c6-cf05-4797-b805-9251e1ebf413 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:17.006684 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35224033-4129-4111-8f36-a082345ba869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:17.007511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35224033-4129-4111-8f36-a082345ba869 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:17.007802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35224033-4129-4111-8f36-a082345ba869 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:17.008083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35224033-4129-4111-8f36-a082345ba869 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:17.009276 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35224033-4129-4111-8f36-a082345ba869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:17.010724 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35224033-4129-4111-8f36-a082345ba869 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:17.012371 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 22/43] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:31:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:17.017478 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35224033-4129-4111-8f36-a082345ba869 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:17.073574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4365c6-cf05-4797-b805-9251e1ebf413 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:17.081604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:17.112539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59772e1b-f17c-4169-9600-9ce678a8e302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:17.113217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59772e1b-f17c-4169-9600-9ce678a8e302 None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:17.113502 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59772e1b-f17c-4169-9600-9ce678a8e302 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:17.113779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59772e1b-f17c-4169-9600-9ce678a8e302 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:17.114530 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59772e1b-f17c-4169-9600-9ce678a8e302 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:17.116226 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35224033-4129-4111-8f36-a082345ba869 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:17.116609 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59772e1b-f17c-4169-9600-9ce678a8e302 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:17.130452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 22/44] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jun 22 11:31:16 2022] POST /identity/v3/roles => generated 223 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:31:17.131246 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59772e1b-f17c-4169-9600-9ce678a8e302 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:17.154433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d5f073-67b7-485c-a6d9-af1c057eff4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:17.155338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5f073-67b7-485c-a6d9-af1c057eff4d None admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:17.155664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5f073-67b7-485c-a6d9-af1c057eff4d None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:17.155947 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d5f073-67b7-485c-a6d9-af1c057eff4d None admin] PATH_INFO: `/v3/domains/default/users/8441761690594b9d8a423147d5d6d62d/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:17.159569 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5f073-67b7-485c-a6d9-af1c057eff4d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8441761690594b9d8a423147d5d6d62d, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:17.162868 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d5f073-67b7-485c-a6d9-af1c057eff4d None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:17.182546 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 23/45] 10.222.0.36 () {58 vars in 1361 bytes} [Wed Jun 22 11:31:17 2022] PUT /identity/v3/domains/default/users/8441761690594b9d8a423147d5d6d62d/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:31:17.362155 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59772e1b-f17c-4169-9600-9ce678a8e302 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:17.399094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13ab3d-378a-4997-9f6f-4e5253ea35e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:17.399808 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13ab3d-378a-4997-9f6f-4e5253ea35e7 None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:17.400098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13ab3d-378a-4997-9f6f-4e5253ea35e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:17.400380 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13ab3d-378a-4997-9f6f-4e5253ea35e7 None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:17.401088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13ab3d-378a-4997-9f6f-4e5253ea35e7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:17.402580 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13ab3d-378a-4997-9f6f-4e5253ea35e7 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:17.429323 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 23/46] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jun 22 11:31:17 2022] POST /identity/v3/projects => generated 312 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:31:19.284606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0c8ce-4959-4b7c-a4f0-64aaf233b40d None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:19.285089 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0c8ce-4959-4b7c-a4f0-64aaf233b40d None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:19.285089 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0c8ce-4959-4b7c-a4f0-64aaf233b40d None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:19.286471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 24/47] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:19.291684 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14061132-0966-42e4-97f4-ef73bbe8a2b1 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:19.292102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14061132-0966-42e4-97f4-ef73bbe8a2b1 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:19.292257 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14061132-0966-42e4-97f4-ef73bbe8a2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:19.309077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-14061132-0966-42e4-97f4-ef73bbe8a2b1 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:19.310271 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-14061132-0966-42e4-97f4-ef73bbe8a2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:19.323294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:19.377421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 24/48] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:19.401065 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0c8ce-4959-4b7c-a4f0-64aaf233b40d None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:19.405009 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:19.478569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c79cf0-2a57-4166-b020-428da7cbb2e6 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:19.478858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c79cf0-2a57-4166-b020-428da7cbb2e6 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:19.479135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c79cf0-2a57-4166-b020-428da7cbb2e6 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:19.480645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 25/49] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:31:19.486545 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4148b-3dd5-453d-b63c-85d875a2214e None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:19.486835 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4148b-3dd5-453d-b63c-85d875a2214e None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:19.487118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4148b-3dd5-453d-b63c-85d875a2214e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:19.506017 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d9d4148b-3dd5-453d-b63c-85d875a2214e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:19.507244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d9d4148b-3dd5-453d-b63c-85d875a2214e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:19.580362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 26/50] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 22 11:31:19.607877 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4148b-3dd5-453d-b63c-85d875a2214e None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:19.611126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0c8ce-4959-4b7c-a4f0-64aaf233b40d None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:19.646830 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c4cfa-100b-4ac1-9c2d-58d360fb06d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:19.647661 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c4cfa-100b-4ac1-9c2d-58d360fb06d0 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:19.647947 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c4cfa-100b-4ac1-9c2d-58d360fb06d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:19.648218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c4cfa-100b-4ac1-9c2d-58d360fb06d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:19.658252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c4cfa-100b-4ac1-9c2d-58d360fb06d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:19.670767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-7a3c4cfa-100b-4ac1-9c2d-58d360fb06d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:31:19.670767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:19.670767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:19.670767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:19.670767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:19.670767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:19.670767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:19.670767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:19.670767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:19.670767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:19.670767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:19.670767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:19.670767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:19.670767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:19.670767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:19.670767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:19.671723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:19.671723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:19.671723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:19.671723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:19.671723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:19.671723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:19.671723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:19.671723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:19.671723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:19.671723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:19.671723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:19.671723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:19.671723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:19.671723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:19.671723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:19.671723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:19.672538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:19.672538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:19.672538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:19.672538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:19.672538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:19.672538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:19.672538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:19.672538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:19.672538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:19.672538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:19.672538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:31:19.672538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:31:19.672538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 25/51] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:31:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:19.679022 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c4cfa-100b-4ac1-9c2d-58d360fb06d0 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:19.686671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:19.710494 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4148b-3dd5-453d-b63c-85d875a2214e None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:19.743379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:19.746502 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d348e-c92b-434c-87eb-c1611a836826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:19.747311 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d348e-c92b-434c-87eb-c1611a836826 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:19.747644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d348e-c92b-434c-87eb-c1611a836826 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:19.747942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d348e-c92b-434c-87eb-c1611a836826 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:19.749145 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d348e-c92b-434c-87eb-c1611a836826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:19.750661 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d348e-c92b-434c-87eb-c1611a836826 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:19.752571 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 27/52] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:31:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:31:19.758827 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d348e-c92b-434c-87eb-c1611a836826 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:19.783798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c4cfa-100b-4ac1-9c2d-58d360fb06d0 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:19.819280 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c46c30-9643-44a8-b1f6-31425dc21cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:19.819997 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c46c30-9643-44a8-b1f6-31425dc21cdc None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:19.820356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c46c30-9643-44a8-b1f6-31425dc21cdc None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:19.820563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c46c30-9643-44a8-b1f6-31425dc21cdc None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:19.821327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c46c30-9643-44a8-b1f6-31425dc21cdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:19.821637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c46c30-9643-44a8-b1f6-31425dc21cdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:19.823026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c46c30-9643-44a8-b1f6-31425dc21cdc None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:19.835496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 26/53] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:31:19.840905 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c46c30-9643-44a8-b1f6-31425dc21cdc None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:19.862954 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d348e-c92b-434c-87eb-c1611a836826 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:19.908756 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929accf0-c7c0-4eda-9c63-294f9059840b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:19.909481 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929accf0-c7c0-4eda-9c63-294f9059840b None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:19.909770 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929accf0-c7c0-4eda-9c63-294f9059840b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:19.910099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929accf0-c7c0-4eda-9c63-294f9059840b None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:19.910891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929accf0-c7c0-4eda-9c63-294f9059840b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:19.912755 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929accf0-c7c0-4eda-9c63-294f9059840b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:19.935008 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:19.950842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 28/54] 10.222.0.36 () {60 vars in 1140 bytes} [Wed Jun 22 11:31:19 2022] POST /identity/v3/projects => generated 298 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:31:19.977047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c46c30-9643-44a8-b1f6-31425dc21cdc None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:20.013208 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:20.026445 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5027924-d4b8-4e3d-a2da-626bb8a14aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:20.027378 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5027924-d4b8-4e3d-a2da-626bb8a14aa8 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:20.027668 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5027924-d4b8-4e3d-a2da-626bb8a14aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:20.027951 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5027924-d4b8-4e3d-a2da-626bb8a14aa8 None admin] PATH_INFO: `/v3/users/8441761690594b9d8a423147d5d6d62d` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:20.029255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5027924-d4b8-4e3d-a2da-626bb8a14aa8 None admin] RBAC: Authorizing `identity:get_user(user_id=8441761690594b9d8a423147d5d6d62d)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:20.033718 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5027924-d4b8-4e3d-a2da-626bb8a14aa8 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:20.034257 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 27/55] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:19 2022] GET /identity/v3/users/8441761690594b9d8a423147d5d6d62d => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:31:20.040901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929accf0-c7c0-4eda-9c63-294f9059840b None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:20.168123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929accf0-c7c0-4eda-9c63-294f9059840b None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:20.207202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad8856-d4b1-4d17-88b2-6a5b6b005453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:20.208294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad8856-d4b1-4d17-88b2-6a5b6b005453 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:20.208797 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad8856-d4b1-4d17-88b2-6a5b6b005453 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:20.209294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad8856-d4b1-4d17-88b2-6a5b6b005453 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:20.210629 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad8856-d4b1-4d17-88b2-6a5b6b005453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:20.212145 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad8856-d4b1-4d17-88b2-6a5b6b005453 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:20.213812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 29/56] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:31:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:20.219803 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5027924-d4b8-4e3d-a2da-626bb8a14aa8 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:20.324173 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5027924-d4b8-4e3d-a2da-626bb8a14aa8 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:20.357928 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9461d-2f2a-448b-b092-6b53bfb4a5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:20.358599 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9461d-2f2a-448b-b092-6b53bfb4a5c2 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:20.358887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9461d-2f2a-448b-b092-6b53bfb4a5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:20.359171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9461d-2f2a-448b-b092-6b53bfb4a5c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:20.360052 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9461d-2f2a-448b-b092-6b53bfb4a5c2 None admin] RBAC: Adding query filter params (role.id=96c584428d924fdba9671949f760c406, scope.domain.id=default, user.id=8441761690594b9d8a423147d5d6d62d) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:20.360374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9461d-2f2a-448b-b092-6b53bfb4a5c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:20.361841 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9461d-2f2a-448b-b092-6b53bfb4a5c2 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:20.370823 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 28/57] 10.222.0.36 () {56 vars in 1322 bytes} [Wed Jun 22 11:31:20 2022] GET /identity/v3/role_assignments?user.id=8441761690594b9d8a423147d5d6d62d&scope.domain.id=default&role.id=96c584428d924fdba9671949f760c406 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:22.079068 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706905e-9bc5-4623-8b80-4d34da014a6d None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:22.079638 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706905e-9bc5-4623-8b80-4d34da014a6d None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:22.079728 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e706905e-9bc5-4623-8b80-4d34da014a6d None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:22.081136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 30/58] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:22.086961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb75f1-e397-41af-998b-5b3288088634 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:22.087400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb75f1-e397-41af-998b-5b3288088634 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:22.087567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb75f1-e397-41af-998b-5b3288088634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:22.103921 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-aadb75f1-e397-41af-998b-5b3288088634 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:22.105119 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-aadb75f1-e397-41af-998b-5b3288088634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:22.156712 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:22.170475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 29/59] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:22.195010 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706905e-9bc5-4623-8b80-4d34da014a6d None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:22.259856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:22.294845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e706905e-9bc5-4623-8b80-4d34da014a6d None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:22.328111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76d40b-4cbc-4267-bb54-a44a2a1a02ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:22.328905 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76d40b-4cbc-4267-bb54-a44a2a1a02ef None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:22.329198 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76d40b-4cbc-4267-bb54-a44a2a1a02ef None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:22.329480 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76d40b-4cbc-4267-bb54-a44a2a1a02ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:22.330655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76d40b-4cbc-4267-bb54-a44a2a1a02ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:22.332131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76d40b-4cbc-4267-bb54-a44a2a1a02ef None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:22.334274 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 31/60] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:31:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:22.339776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb75f1-e397-41af-998b-5b3288088634 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:22.434771 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb75f1-e397-41af-998b-5b3288088634 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:22.467940 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330e382-2929-4a0a-b163-3f85bc699c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:22.468535 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330e382-2929-4a0a-b163-3f85bc699c5a None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:22.468820 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330e382-2929-4a0a-b163-3f85bc699c5a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:22.469100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330e382-2929-4a0a-b163-3f85bc699c5a None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:22.469805 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330e382-2929-4a0a-b163-3f85bc699c5a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:22.471268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330e382-2929-4a0a-b163-3f85bc699c5a None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:22.496814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 30/61] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jun 22 11:31:22 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:31:25.519724 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c233ea4-a375-45ee-93e3-afb73c960b77 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:25.520230 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c233ea4-a375-45ee-93e3-afb73c960b77 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:25.520304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c233ea4-a375-45ee-93e3-afb73c960b77 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:25.522291 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 32/62] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:25.527863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c51c04-ae9a-4e70-843c-ae081fd9b49d None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:25.528339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c51c04-ae9a-4e70-843c-ae081fd9b49d None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:25.528415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c51c04-ae9a-4e70-843c-ae081fd9b49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:25.549926 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-e0c51c04-ae9a-4e70-843c-ae081fd9b49d None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:25.551136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e0c51c04-ae9a-4e70-843c-ae081fd9b49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:25.596548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:25.635806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 31/63] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:25.660177 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c233ea4-a375-45ee-93e3-afb73c960b77 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:25.803066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c233ea4-a375-45ee-93e3-afb73c960b77 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:25.824937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deace2fe-d6db-4551-ba69-c9ffa7024d1f None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:25.825587 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deace2fe-d6db-4551-ba69-c9ffa7024d1f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:25.826083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deace2fe-d6db-4551-ba69-c9ffa7024d1f None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:25.827702 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 32/64] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:31:25.845384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e53a05-e636-48f3-82a2-b5df9138fb02 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:25.845664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e53a05-e636-48f3-82a2-b5df9138fb02 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:25.845952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e53a05-e636-48f3-82a2-b5df9138fb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:25.854874 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3e951-2a1e-451a-820f-b69920e3d25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:25.855716 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3e951-2a1e-451a-820f-b69920e3d25e None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:25.856005 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3e951-2a1e-451a-820f-b69920e3d25e None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:25.856289 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3e951-2a1e-451a-820f-b69920e3d25e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:25.858618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:25.869147 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-10e53a05-e636-48f3-82a2-b5df9138fb02 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:25.870716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-10e53a05-e636-48f3-82a2-b5df9138fb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:25.887807 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3e951-2a1e-451a-820f-b69920e3d25e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:25.896038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-0ec3e951-2a1e-451a-820f-b69920e3d25e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:25.896038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:25.896038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:25.896038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:25.896038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:25.896038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:25.896038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:25.896038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:25.896038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:25.896038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:25.896038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:25.896038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:25.896038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:25.896038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:25.896038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:25.896038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:25.897011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:25.897011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:25.897011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:25.897011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:25.897011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:25.897011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:25.897011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:25.897011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:25.897011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:25.897011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:25.897011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:25.897011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:25.897011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:25.897011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:25.897011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:25.897011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:25.897813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:25.897813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:25.897813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:25.897813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:25.897813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:25.897813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:25.897813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:25.897813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:25.897813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:25.897813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:25.897813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:25.897813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:25.900233 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 33/65] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:25.908350 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3e951-2a1e-451a-820f-b69920e3d25e None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:25.995937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 33/66] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:25.998020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db037ee-cd4a-4574-9822-8022915dcc2b None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:25.998460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db037ee-cd4a-4574-9822-8022915dcc2b None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:25.998884 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db037ee-cd4a-4574-9822-8022915dcc2b None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:26.000355 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 34/67] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:26.006212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e50c86-7f8c-4797-b629-c9b0a126b304 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:26.006839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e50c86-7f8c-4797-b629-c9b0a126b304 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:26.007269 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e50c86-7f8c-4797-b629-c9b0a126b304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:26.036047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-19e50c86-7f8c-4797-b629-c9b0a126b304 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:26.038701 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-19e50c86-7f8c-4797-b629-c9b0a126b304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:26.073486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3e951-2a1e-451a-820f-b69920e3d25e None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:26.110316 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be46a6-dd62-41cf-8699-9364eeb30022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:26.111255 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be46a6-dd62-41cf-8699-9364eeb30022 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:26.111744 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be46a6-dd62-41cf-8699-9364eeb30022 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:26.112175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be46a6-dd62-41cf-8699-9364eeb30022 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:26.113099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be46a6-dd62-41cf-8699-9364eeb30022 None admin] RBAC: Adding query filter params (name=member) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:26.113552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be46a6-dd62-41cf-8699-9364eeb30022 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:26.115718 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be46a6-dd62-41cf-8699-9364eeb30022 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:26.125180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 34/68] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:26.126226 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be46a6-dd62-41cf-8699-9364eeb30022 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:26.151449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 35/69] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:26.152563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e50c86-7f8c-4797-b629-c9b0a126b304 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:26.210767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:26.217308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:26.260402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be46a6-dd62-41cf-8699-9364eeb30022 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:26.294067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e50c86-7f8c-4797-b629-c9b0a126b304 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:26.300553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b4b34-a66e-457e-a727-f779793e02a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:26.301416 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b4b34-a66e-457e-a727-f779793e02a1 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:26.301716 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b4b34-a66e-457e-a727-f779793e02a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:26.302006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b4b34-a66e-457e-a727-f779793e02a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:26.315044 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b4b34-a66e-457e-a727-f779793e02a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:26.325424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-f30b4b34-a66e-457e-a727-f779793e02a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:31:26.325424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:26.325424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:26.325424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:26.325424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:26.325424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:26.325424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:26.325424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:26.325424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:26.325424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:26.325424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:26.325424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:26.325424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:26.325424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:26.325424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:26.325424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:26.326371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:26.326371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:26.326371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:26.326371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:26.326371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:26.326371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:26.326371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:26.326371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:26.326371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:26.326371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:26.326371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:26.326371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:26.326371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:26.326371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:26.326371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:26.326371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:26.327273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:26.327273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:26.327273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:26.327273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:26.327273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:26.327273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:26.327273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:26.327273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:26.327273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:26.327273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:26.327273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:31:26.327273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:26.329348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 35/70] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:31:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:26.330375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b4b34-a66e-457e-a727-f779793e02a1 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:26.339889 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294bb37-7ff2-4ca8-aa1d-92e678d5763e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:26.340783 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294bb37-7ff2-4ca8-aa1d-92e678d5763e None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:26.341071 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294bb37-7ff2-4ca8-aa1d-92e678d5763e None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:26.341353 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294bb37-7ff2-4ca8-aa1d-92e678d5763e None admin] PATH_INFO: `/v3/users/d6d9a2f332cf40a6b5bc0d534f1eab4b` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:26.358559 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294bb37-7ff2-4ca8-aa1d-92e678d5763e None admin] RBAC: Authorizing `identity:get_user(user_id=d6d9a2f332cf40a6b5bc0d534f1eab4b)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:26.360103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294bb37-7ff2-4ca8-aa1d-92e678d5763e None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:26.361856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 36/71] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:26 2022] GET /identity/v3/users/d6d9a2f332cf40a6b5bc0d534f1eab4b => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:26.363589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb5d74-190a-4752-a2ce-19b85c237bed None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:26.363888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb5d74-190a-4752-a2ce-19b85c237bed None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:26.364174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb5d74-190a-4752-a2ce-19b85c237bed None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:26.366184 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 37/72] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:26.367136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb5d74-190a-4752-a2ce-19b85c237bed None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:26.442220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b4b34-a66e-457e-a727-f779793e02a1 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:26.458537 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:26.463571 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:26.473851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb5d74-190a-4752-a2ce-19b85c237bed None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:26.476633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4033e74-201c-4371-abff-990c7f228a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:26.477427 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4033e74-201c-4371-abff-990c7f228a74 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:26.477712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4033e74-201c-4371-abff-990c7f228a74 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:26.477990 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4033e74-201c-4371-abff-990c7f228a74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:26.488070 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4033e74-201c-4371-abff-990c7f228a74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:26.496109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-b4033e74-201c-4371-abff-990c7f228a74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:31:26.496109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:26.496109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:26.496109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:26.496109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:26.496109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:26.496109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:26.496109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:26.496109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:26.496109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:26.496109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:26.496109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:26.496109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:26.496109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:26.496109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:26.496109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:26.497031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:26.497031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:26.497031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:26.497031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:26.497031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:26.497031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:26.497031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:26.497031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:26.497031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:26.497031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:26.497031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:26.497031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:26.497031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:26.497031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:26.497031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:26.497031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:26.497828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:26.497828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:26.497828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:26.497828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:26.497828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:26.497828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:26.497828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:26.497828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:26.497828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:26.497828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:26.497828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:31:26.497828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:26.497828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 36/73] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:31:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:26.498719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61e2d5-3500-4a85-a3ab-f34b2be96a0c None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:26.498782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61e2d5-3500-4a85-a3ab-f34b2be96a0c None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:26.499026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61e2d5-3500-4a85-a3ab-f34b2be96a0c None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:26.500497 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 37/74] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:26.501045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61e2d5-3500-4a85-a3ab-f34b2be96a0c None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:26.514191 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d943e22-be56-4aac-97ad-2c2d276073ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:26.515312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d943e22-be56-4aac-97ad-2c2d276073ac None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:26.515571 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d943e22-be56-4aac-97ad-2c2d276073ac None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:26.515865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d943e22-be56-4aac-97ad-2c2d276073ac None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:26.516760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d943e22-be56-4aac-97ad-2c2d276073ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:26.517169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d943e22-be56-4aac-97ad-2c2d276073ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:26.518468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d943e22-be56-4aac-97ad-2c2d276073ac None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:26.527512 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 38/75] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:31:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:26.529121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472756d9-eb42-46f9-86a6-e2607aaad894 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:26.529666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472756d9-eb42-46f9-86a6-e2607aaad894 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:26.529738 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472756d9-eb42-46f9-86a6-e2607aaad894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:26.550036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-472756d9-eb42-46f9-86a6-e2607aaad894 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:26.551265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-472756d9-eb42-46f9-86a6-e2607aaad894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:26.615448 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61e2d5-3500-4a85-a3ab-f34b2be96a0c None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:26.620928 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 39/76] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:26.621783 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472756d9-eb42-46f9-86a6-e2607aaad894 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:26.650316 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81874d-ed3d-421c-ad28-c80ce4259c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:26.650969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81874d-ed3d-421c-ad28-c80ce4259c09 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:26.651253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81874d-ed3d-421c-ad28-c80ce4259c09 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:26.651565 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81874d-ed3d-421c-ad28-c80ce4259c09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:26.696214 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:26.707229 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:26.719904 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a81874d-ed3d-421c-ad28-c80ce4259c09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:26.721425 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a81874d-ed3d-421c-ad28-c80ce4259c09 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:26.723022 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472756d9-eb42-46f9-86a6-e2607aaad894 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:26.757678 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d63a06-7cd3-4655-84ee-965c86636bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:26.758348 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d63a06-7cd3-4655-84ee-965c86636bc1 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:26.758633 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d63a06-7cd3-4655-84ee-965c86636bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:26.758909 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d63a06-7cd3-4655-84ee-965c86636bc1 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:26.759688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d63a06-7cd3-4655-84ee-965c86636bc1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:26.759993 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d63a06-7cd3-4655-84ee-965c86636bc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:26.761392 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d63a06-7cd3-4655-84ee-965c86636bc1 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:26.769795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 40/77] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:26.771562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685cedf-70be-41f6-9e15-319c1468e356 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:26.771852 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685cedf-70be-41f6-9e15-319c1468e356 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:26.772133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685cedf-70be-41f6-9e15-319c1468e356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:26.790186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-2685cedf-70be-41f6-9e15-319c1468e356 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:26.791389 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2685cedf-70be-41f6-9e15-319c1468e356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:26.822414 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 38/78] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:31:26.823269 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81874d-ed3d-421c-ad28-c80ce4259c09 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:26.859381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 41/79] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:26.860251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685cedf-70be-41f6-9e15-319c1468e356 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:26.910990 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:26.935553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81874d-ed3d-421c-ad28-c80ce4259c09 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:26.943393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:26.967681 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685cedf-70be-41f6-9e15-319c1468e356 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:26.969954 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fb832-d1ac-4f60-b9d6-2ae4bd8cfbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:26.970836 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fb832-d1ac-4f60-b9d6-2ae4bd8cfbee None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:26.971126 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fb832-d1ac-4f60-b9d6-2ae4bd8cfbee None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:26.971544 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fb832-d1ac-4f60-b9d6-2ae4bd8cfbee None admin] PATH_INFO: `/v3/users/d6d9a2f332cf40a6b5bc0d534f1eab4b` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:26.972655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fb832-d1ac-4f60-b9d6-2ae4bd8cfbee None admin] RBAC: Authorizing `identity:get_user(user_id=d6d9a2f332cf40a6b5bc0d534f1eab4b)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:26.974687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fb832-d1ac-4f60-b9d6-2ae4bd8cfbee None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:26.976369 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 39/80] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:26 2022] GET /identity/v3/users/d6d9a2f332cf40a6b5bc0d534f1eab4b => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:26.977141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fb832-d1ac-4f60-b9d6-2ae4bd8cfbee None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:27.002635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301bdc2-bb2c-4852-9423-9e53ca89021e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:27.003622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301bdc2-bb2c-4852-9423-9e53ca89021e None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:27.003969 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301bdc2-bb2c-4852-9423-9e53ca89021e None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:27.004293 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301bdc2-bb2c-4852-9423-9e53ca89021e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:27.018957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301bdc2-bb2c-4852-9423-9e53ca89021e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:27.033190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-e301bdc2-bb2c-4852-9423-9e53ca89021e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:27.033190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:27.033190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:27.033190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:27.033190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:27.033190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:27.033190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:27.033190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:27.033190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:27.033190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:27.033190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:27.033190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:27.033190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:27.033190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:27.033190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:27.033190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:27.034135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:27.034135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:27.034135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:27.034135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:27.034135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:27.034135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:27.034135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:27.034135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:27.034135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:27.034135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:27.034135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:27.034135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:27.034135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:27.034135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:27.034135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:27.034135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:27.034876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:27.034876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:27.034876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:27.034876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:27.034876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:27.034876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:27.034876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:27.034876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:27.034876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:27.034876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:27.034876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:27.034876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:31:27.041153 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 42/81] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:27.041153 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301bdc2-bb2c-4852-9423-9e53ca89021e None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:27.098911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fb832-d1ac-4f60-b9d6-2ae4bd8cfbee None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:27.135021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b209e9e-c050-4b14-9f5a-f343bbc486cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:27.135930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b209e9e-c050-4b14-9f5a-f343bbc486cd None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:27.136213 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b209e9e-c050-4b14-9f5a-f343bbc486cd None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:27.136490 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b209e9e-c050-4b14-9f5a-f343bbc486cd None admin] PATH_INFO: `/v3/users/8441761690594b9d8a423147d5d6d62d` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:27.137684 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b209e9e-c050-4b14-9f5a-f343bbc486cd None admin] RBAC: Authorizing `identity:get_user(user_id=8441761690594b9d8a423147d5d6d62d)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:27.139174 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b209e9e-c050-4b14-9f5a-f343bbc486cd None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:27.141108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 40/82] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:26 2022] GET /identity/v3/users/8441761690594b9d8a423147d5d6d62d => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:27.141869 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b209e9e-c050-4b14-9f5a-f343bbc486cd None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:27.151406 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301bdc2-bb2c-4852-9423-9e53ca89021e None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:27.156742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:27.185832 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6bb5a-c594-4424-a901-abce972c458b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:27.186631 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6bb5a-c594-4424-a901-abce972c458b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:27.186918 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6bb5a-c594-4424-a901-abce972c458b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:27.187194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6bb5a-c594-4424-a901-abce972c458b None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:27.188450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6bb5a-c594-4424-a901-abce972c458b None admin] RBAC: Authorizing `identity:get_project(project_id=e263f641bede4564a5855da9ec8514ed)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:27.189923 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6bb5a-c594-4424-a901-abce972c458b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:27.191679 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 43/83] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:27 2022] GET /identity/v3/projects/e263f641bede4564a5855da9ec8514ed => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:27.195953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6bb5a-c594-4424-a901-abce972c458b None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:27.199433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:27.245972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b209e9e-c050-4b14-9f5a-f343bbc486cd None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:27.286104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877cf7f-5de5-4b05-8a17-b22689a934eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:27.287682 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877cf7f-5de5-4b05-8a17-b22689a934eb None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:27.287962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877cf7f-5de5-4b05-8a17-b22689a934eb None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:27.288240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877cf7f-5de5-4b05-8a17-b22689a934eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:27.289424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877cf7f-5de5-4b05-8a17-b22689a934eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:27.290884 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877cf7f-5de5-4b05-8a17-b22689a934eb None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:27.292537 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 41/84] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:31:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:27.293296 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877cf7f-5de5-4b05-8a17-b22689a934eb None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:27.299449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6bb5a-c594-4424-a901-abce972c458b None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:27.333210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35309ce0-a535-411b-a800-0a45db803d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:27.333962 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35309ce0-a535-411b-a800-0a45db803d86 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:27.334198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35309ce0-a535-411b-a800-0a45db803d86 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:27.334483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35309ce0-a535-411b-a800-0a45db803d86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:27.375868 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:27.395734 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877cf7f-5de5-4b05-8a17-b22689a934eb None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:27.407737 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35309ce0-a535-411b-a800-0a45db803d86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:27.409218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35309ce0-a535-411b-a800-0a45db803d86 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:27.412465 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:27.435657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8b60e-3cb1-492d-8994-20f6ffea5706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:27.436579 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8b60e-3cb1-492d-8994-20f6ffea5706 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:27.436842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8b60e-3cb1-492d-8994-20f6ffea5706 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:27.437135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8b60e-3cb1-492d-8994-20f6ffea5706 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:27.437908 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8b60e-3cb1-492d-8994-20f6ffea5706 None admin] RBAC: Adding query filter params (name=member) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:27.438227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8b60e-3cb1-492d-8994-20f6ffea5706 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:27.439701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8b60e-3cb1-492d-8994-20f6ffea5706 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:27.448751 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 42/85] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:27.449482 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8b60e-3cb1-492d-8994-20f6ffea5706 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:27.531281 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 44/86] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:31:27.532188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35309ce0-a535-411b-a800-0a45db803d86 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:27.564779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8b60e-3cb1-492d-8994-20f6ffea5706 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:27.598992 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1872286-a2ce-4827-bbdd-b064ed481271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:27.600208 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1872286-a2ce-4827-bbdd-b064ed481271 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:27.600499 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1872286-a2ce-4827-bbdd-b064ed481271 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:27.600783 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1872286-a2ce-4827-bbdd-b064ed481271 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:27.613089 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:27.633341 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35309ce0-a535-411b-a800-0a45db803d86 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:27.636509 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:27.669082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1872286-a2ce-4827-bbdd-b064ed481271 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:27.670376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52886e8d-cc57-4931-b558-8be106f7ad78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:27.670575 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1872286-a2ce-4827-bbdd-b064ed481271 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:27.671029 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52886e8d-cc57-4931-b558-8be106f7ad78 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:27.671312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52886e8d-cc57-4931-b558-8be106f7ad78 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:27.671618 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52886e8d-cc57-4931-b558-8be106f7ad78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:27.672442 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52886e8d-cc57-4931-b558-8be106f7ad78 None admin] RBAC: Adding query filter params (role.id=f2336919c5104f6a8210e99994f4d455, scope.project.id=e263f641bede4564a5855da9ec8514ed, user.id=d6d9a2f332cf40a6b5bc0d534f1eab4b) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:27.672753 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52886e8d-cc57-4931-b558-8be106f7ad78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:27.674189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52886e8d-cc57-4931-b558-8be106f7ad78 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:27.679651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 45/87] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:31:27 2022] GET /identity/v3/role_assignments?user.id=d6d9a2f332cf40a6b5bc0d534f1eab4b&scope.project.id=e263f641bede4564a5855da9ec8514ed&role.id=f2336919c5104f6a8210e99994f4d455 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:27.680446 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52886e8d-cc57-4931-b558-8be106f7ad78 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:27.795841 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52886e8d-cc57-4931-b558-8be106f7ad78 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:27.807378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 43/88] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:31:27.807378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1872286-a2ce-4827-bbdd-b064ed481271 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:27.844078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e8aee-9c9b-421f-9cff-a950d8de6c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:27.844761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e8aee-9c9b-421f-9cff-a950d8de6c34 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:27.845065 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e8aee-9c9b-421f-9cff-a950d8de6c34 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:27.845330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e8aee-9c9b-421f-9cff-a950d8de6c34 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:27.846059 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e8aee-9c9b-421f-9cff-a950d8de6c34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:27.847617 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e8aee-9c9b-421f-9cff-a950d8de6c34 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:27.860985 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:27.879380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 46/89] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jun 22 11:31:27 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:31:27.879380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e8aee-9c9b-421f-9cff-a950d8de6c34 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:27.901767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:27.956564 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1872286-a2ce-4827-bbdd-b064ed481271 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:28.007215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf6ff5-29cc-4920-b227-808b131f6169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:28.008184 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e8aee-9c9b-421f-9cff-a950d8de6c34 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:28.008372 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf6ff5-29cc-4920-b227-808b131f6169 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:28.008760 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf6ff5-29cc-4920-b227-808b131f6169 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:28.009139 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf6ff5-29cc-4920-b227-808b131f6169 None admin] PATH_INFO: `/v3/users/d6d9a2f332cf40a6b5bc0d534f1eab4b` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:28.011046 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf6ff5-29cc-4920-b227-808b131f6169 None admin] RBAC: Authorizing `identity:get_user(user_id=d6d9a2f332cf40a6b5bc0d534f1eab4b)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:28.013903 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf6ff5-29cc-4920-b227-808b131f6169 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:28.016121 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 44/90] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:27 2022] GET /identity/v3/users/d6d9a2f332cf40a6b5bc0d534f1eab4b => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:28.017078 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf6ff5-29cc-4920-b227-808b131f6169 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:28.059672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a310c1-4d16-4c1f-a573-966600115051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:28.060479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a310c1-4d16-4c1f-a573-966600115051 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:28.060769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a310c1-4d16-4c1f-a573-966600115051 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:28.061053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a310c1-4d16-4c1f-a573-966600115051 None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:28.062262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a310c1-4d16-4c1f-a573-966600115051 None admin] RBAC: Authorizing `identity:get_project(project_id=e263f641bede4564a5855da9ec8514ed)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:28.064762 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a310c1-4d16-4c1f-a573-966600115051 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:28.065639 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 47/91] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:27 2022] GET /identity/v3/projects/e263f641bede4564a5855da9ec8514ed => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:28.070768 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a310c1-4d16-4c1f-a573-966600115051 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:28.143997 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf6ff5-29cc-4920-b227-808b131f6169 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:28.159739 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:28.188540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:28.200947 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a310c1-4d16-4c1f-a573-966600115051 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:28.205557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509412f-624b-4565-9a77-ef3da53e5dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:28.206452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509412f-624b-4565-9a77-ef3da53e5dba None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:28.206742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509412f-624b-4565-9a77-ef3da53e5dba None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:28.207030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509412f-624b-4565-9a77-ef3da53e5dba None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:28.208362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509412f-624b-4565-9a77-ef3da53e5dba None admin] RBAC: Authorizing `identity:get_project(project_id=e263f641bede4564a5855da9ec8514ed)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:28.210350 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509412f-624b-4565-9a77-ef3da53e5dba None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:28.213005 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 45/92] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:28 2022] GET /identity/v3/projects/e263f641bede4564a5855da9ec8514ed => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:28.214325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509412f-624b-4565-9a77-ef3da53e5dba None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:28.250007 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b2876-e2ad-481c-bb1a-99de60017715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:28.250776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b2876-e2ad-481c-bb1a-99de60017715 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:28.251099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b2876-e2ad-481c-bb1a-99de60017715 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:28.251503 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b2876-e2ad-481c-bb1a-99de60017715 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:28.352551 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b2876-e2ad-481c-bb1a-99de60017715 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:28.354293 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b2876-e2ad-481c-bb1a-99de60017715 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:28.362969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509412f-624b-4565-9a77-ef3da53e5dba None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:28.401085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc6f1b-bf33-4670-9f05-dcee8215d37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:28.401754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc6f1b-bf33-4670-9f05-dcee8215d37f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:28.402041 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc6f1b-bf33-4670-9f05-dcee8215d37f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:28.402321 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc6f1b-bf33-4670-9f05-dcee8215d37f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:28.403280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc6f1b-bf33-4670-9f05-dcee8215d37f None admin] RBAC: Adding query filter params (role.id=b4be7021e4104740b8b0b3b40ecd3d59, scope.project.id=e263f641bede4564a5855da9ec8514ed, user.id=d6d9a2f332cf40a6b5bc0d534f1eab4b) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:28.403628 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc6f1b-bf33-4670-9f05-dcee8215d37f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:28.405236 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc6f1b-bf33-4670-9f05-dcee8215d37f None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:28.411996 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 46/93] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:31:28 2022] GET /identity/v3/role_assignments?user.id=d6d9a2f332cf40a6b5bc0d534f1eab4b&scope.project.id=e263f641bede4564a5855da9ec8514ed&role.id=b4be7021e4104740b8b0b3b40ecd3d59 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:28.412584 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc6f1b-bf33-4670-9f05-dcee8215d37f None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:28.455566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:28.464731 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:28.467037 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 48/94] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:31:28.473164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b2876-e2ad-481c-bb1a-99de60017715 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:28.531674 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc6f1b-bf33-4670-9f05-dcee8215d37f None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:28.580517 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0a6b1-dd96-4c93-810b-bb149dcf922d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:28.581198 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0a6b1-dd96-4c93-810b-bb149dcf922d None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:28.581490 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0a6b1-dd96-4c93-810b-bb149dcf922d None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:28.581770 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e0a6b1-dd96-4c93-810b-bb149dcf922d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:28.582636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0a6b1-dd96-4c93-810b-bb149dcf922d None admin] RBAC: Adding query filter params (role.id=96c584428d924fdba9671949f760c406, scope.project.id=e263f641bede4564a5855da9ec8514ed, user.id=8441761690594b9d8a423147d5d6d62d) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:28.582950 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0a6b1-dd96-4c93-810b-bb149dcf922d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:28.585044 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e0a6b1-dd96-4c93-810b-bb149dcf922d None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:28.609452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 47/95] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:31:28 2022] GET /identity/v3/role_assignments?user.id=8441761690594b9d8a423147d5d6d62d&scope.project.id=e263f641bede4564a5855da9ec8514ed&role.id=96c584428d924fdba9671949f760c406 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:28.648802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b2876-e2ad-481c-bb1a-99de60017715 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:28.687928 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e860743-c0b5-4574-a2eb-0c50da59b97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:28.688831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e860743-c0b5-4574-a2eb-0c50da59b97d None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:28.689121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e860743-c0b5-4574-a2eb-0c50da59b97d None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:28.689402 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e860743-c0b5-4574-a2eb-0c50da59b97d None admin] PATH_INFO: `/v3/projects/0ce6633b03a947ecb728e95eea7a60b5` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:28.690730 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e860743-c0b5-4574-a2eb-0c50da59b97d None admin] RBAC: Authorizing `identity:get_project(project_id=0ce6633b03a947ecb728e95eea7a60b5)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:28.692320 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e860743-c0b5-4574-a2eb-0c50da59b97d None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:28.699379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 49/96] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:28 2022] GET /identity/v3/projects/0ce6633b03a947ecb728e95eea7a60b5 => generated 312 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:31:28.701672 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0a6b1-dd96-4c93-810b-bb149dcf922d None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:28.847941 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e0a6b1-dd96-4c93-810b-bb149dcf922d None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:28.862154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:28.904587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4d7d6-66f3-4b0a-aa9e-34451036e5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:28.905268 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4d7d6-66f3-4b0a-aa9e-34451036e5fa None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:28.905554 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4d7d6-66f3-4b0a-aa9e-34451036e5fa None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:28.905835 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4d7d6-66f3-4b0a-aa9e-34451036e5fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:28.906681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e4d7d6-66f3-4b0a-aa9e-34451036e5fa None admin] RBAC: Adding query filter params (role.id=f2336919c5104f6a8210e99994f4d455, scope.project.id=0ce6633b03a947ecb728e95eea7a60b5, user.id=d6d9a2f332cf40a6b5bc0d534f1eab4b) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:28.906999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e4d7d6-66f3-4b0a-aa9e-34451036e5fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:28.908597 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e4d7d6-66f3-4b0a-aa9e-34451036e5fa None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:28.920215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 48/97] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:31:28 2022] GET /identity/v3/role_assignments?user.id=d6d9a2f332cf40a6b5bc0d534f1eab4b&scope.project.id=0ce6633b03a947ecb728e95eea7a60b5&role.id=f2336919c5104f6a8210e99994f4d455 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:30.092897 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cfe40-5179-4c88-8fa3-7179ebe91dd1 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:30.093917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cfe40-5179-4c88-8fa3-7179ebe91dd1 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:30.094408 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cfe40-5179-4c88-8fa3-7179ebe91dd1 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:30.096296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 50/98] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:30.102578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2a7f2-1043-4101-80d7-74e6a7012d0a None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:30.103452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2a7f2-1043-4101-80d7-74e6a7012d0a None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:30.103934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2a7f2-1043-4101-80d7-74e6a7012d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:30.124972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-71b2a7f2-1043-4101-80d7-74e6a7012d0a None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:30.126612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-71b2a7f2-1043-4101-80d7-74e6a7012d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:30.216584 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 49/99] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:30.250940 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13cfe40-5179-4c88-8fa3-7179ebe91dd1 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:30.338537 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:30.396565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13cfe40-5179-4c88-8fa3-7179ebe91dd1 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:30.440041 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6483cb-597e-4564-bd69-f5d0589ef87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:30.440848 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6483cb-597e-4564-bd69-f5d0589ef87c None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:30.441137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6483cb-597e-4564-bd69-f5d0589ef87c None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:30.441419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6483cb-597e-4564-bd69-f5d0589ef87c None admin] PATH_INFO: `/v3/roles/member` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:30.456204 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6483cb-597e-4564-bd69-f5d0589ef87c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:30.464400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-cb6483cb-597e-4564-bd69-f5d0589ef87c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:30.464400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:30.464400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:30.464400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:30.464400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:30.464400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:30.464400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:30.464400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:30.464400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:30.464400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:30.464400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:30.464400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:30.464400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:30.464400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:30.464400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:30.464400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:30.465227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:30.465227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:30.465227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:30.465227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:30.465227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:30.465227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:30.465227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:30.465227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:30.465227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:30.465227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:30.465227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:30.465227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:30.465227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:30.465227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:30.465227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:30.465227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:30.465984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:30.465984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:30.465984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:30.465984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:30.465984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:30.465984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:30.465984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:30.465984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:30.465984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:30.465984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:30.465984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:30.465984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:31:30.465984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 51/100] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:30.472946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2a7f2-1043-4101-80d7-74e6a7012d0a None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:30.608774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:30.612177 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2a7f2-1043-4101-80d7-74e6a7012d0a None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:30.664612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1e7d0-c402-49dd-9fad-e8fc6fd52b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:30.665297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1e7d0-c402-49dd-9fad-e8fc6fd52b70 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:30.665582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1e7d0-c402-49dd-9fad-e8fc6fd52b70 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:30.665860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1e7d0-c402-49dd-9fad-e8fc6fd52b70 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:30.666623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1e7d0-c402-49dd-9fad-e8fc6fd52b70 None admin] RBAC: Adding query filter params (name=member) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:30.666931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1e7d0-c402-49dd-9fad-e8fc6fd52b70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:30.668356 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1e7d0-c402-49dd-9fad-e8fc6fd52b70 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:30.677440 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 50/101] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:30.683206 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6483cb-597e-4564-bd69-f5d0589ef87c None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:30.821199 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6483cb-597e-4564-bd69-f5d0589ef87c None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:30.855769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:30.867632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971473be-27a8-453d-b542-373a649915d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:30.868611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971473be-27a8-453d-b542-373a649915d4 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:30.868898 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971473be-27a8-453d-b542-373a649915d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:30.869179 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971473be-27a8-453d-b542-373a649915d4 None admin] PATH_INFO: `/v3/users/d6d9a2f332cf40a6b5bc0d534f1eab4b` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:30.870681 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971473be-27a8-453d-b542-373a649915d4 None admin] RBAC: Authorizing `identity:get_user(user_id=d6d9a2f332cf40a6b5bc0d534f1eab4b)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:30.872198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971473be-27a8-453d-b542-373a649915d4 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:30.874035 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 52/102] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:30 2022] GET /identity/v3/users/d6d9a2f332cf40a6b5bc0d534f1eab4b => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:30.879483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1e7d0-c402-49dd-9fad-e8fc6fd52b70 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:30.893419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0bdcd-a649-4057-9222-54a483e1ca05 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:30.893702 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0bdcd-a649-4057-9222-54a483e1ca05 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:30.894242 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0bdcd-a649-4057-9222-54a483e1ca05 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:30.896339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 53/103] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:31:30.901559 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f157e19-04a3-4ccf-b64b-26815db6f391 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:30.902112 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f157e19-04a3-4ccf-b64b-26815db6f391 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:30.902534 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f157e19-04a3-4ccf-b64b-26815db6f391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:30.923847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-3f157e19-04a3-4ccf-b64b-26815db6f391 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:30.926494 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-3f157e19-04a3-4ccf-b64b-26815db6f391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:31.029516 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 54/104] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 22 11:31:31.031983 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1e7d0-c402-49dd-9fad-e8fc6fd52b70 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:31.072214 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810f26c-89fb-4cb7-acc0-63fb6fe32d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:31.072759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f157e19-04a3-4ccf-b64b-26815db6f391 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:31.073427 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810f26c-89fb-4cb7-acc0-63fb6fe32d23 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:31.073717 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810f26c-89fb-4cb7-acc0-63fb6fe32d23 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:31.074001 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810f26c-89fb-4cb7-acc0-63fb6fe32d23 None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:31.075247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6810f26c-89fb-4cb7-acc0-63fb6fe32d23 None admin] RBAC: Authorizing `identity:get_project(project_id=e263f641bede4564a5855da9ec8514ed)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:31.078606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6810f26c-89fb-4cb7-acc0-63fb6fe32d23 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:31.078606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 51/105] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:30 2022] GET /identity/v3/projects/e263f641bede4564a5855da9ec8514ed => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:31.095031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810f26c-89fb-4cb7-acc0-63fb6fe32d23 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:31.131751 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:31.236757 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f157e19-04a3-4ccf-b64b-26815db6f391 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:31.253372 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810f26c-89fb-4cb7-acc0-63fb6fe32d23 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:31.259099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:31.279195 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656dc4ab-1ea6-4a61-b108-f28b54335ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:31.280349 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656dc4ab-1ea6-4a61-b108-f28b54335ff5 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:31.280854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656dc4ab-1ea6-4a61-b108-f28b54335ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:31.281324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656dc4ab-1ea6-4a61-b108-f28b54335ff5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:31.294024 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656dc4ab-1ea6-4a61-b108-f28b54335ff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:31.303934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-656dc4ab-1ea6-4a61-b108-f28b54335ff5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 22 11:31:31.303934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:31.303934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:31.303934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:31.303934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:31.303934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:31.303934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:31.303934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:31.303934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:31.303934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:31.303934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:31.303934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:31.303934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:31.303934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:31.303934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:31.303934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:31.304826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:31.304826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:31.304826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:31.304826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:31.304826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:31.304826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:31.304826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:31.304826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:31.304826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:31.304826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:31.304826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:31.304826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:31.304826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:31.304826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:31.304826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:31.304826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:31.305600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:31.305600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:31.305600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:31.305600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:31.305600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:31.305600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:31.305600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:31.305600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:31.305600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:31.305600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:31.305600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 22 11:31:31.305600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:31:31.308754 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 55/106] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:31:31.315261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656dc4ab-1ea6-4a61-b108-f28b54335ff5 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:31.328258 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944c41e-2fef-45f7-b4ee-1625ee83223f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:31.329101 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944c41e-2fef-45f7-b4ee-1625ee83223f None admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:31.329390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944c41e-2fef-45f7-b4ee-1625ee83223f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:31.329676 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944c41e-2fef-45f7-b4ee-1625ee83223f None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed/users/d6d9a2f332cf40a6b5bc0d534f1eab4b/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:31.334692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944c41e-2fef-45f7-b4ee-1625ee83223f None admin] RBAC: Authorizing `identity:create_grant(project_id=e263f641bede4564a5855da9ec8514ed, user_id=d6d9a2f332cf40a6b5bc0d534f1eab4b, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:31.336209 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944c41e-2fef-45f7-b4ee-1625ee83223f None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:31.371375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 52/107] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:31:31 2022] PUT /identity/v3/projects/e263f641bede4564a5855da9ec8514ed/users/d6d9a2f332cf40a6b5bc0d534f1eab4b/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:31:31.443930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656dc4ab-1ea6-4a61-b108-f28b54335ff5 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:31.486925 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e8ba8-d68b-4a26-a3f8-f99301647e71 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:31.487222 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e8ba8-d68b-4a26-a3f8-f99301647e71 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:31.487536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98e8ba8-d68b-4a26-a3f8-f99301647e71 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:31.490056 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 53/108] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:31.494670 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb3e01-24b8-4ab3-bc3d-f58a33379a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:31.495624 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7238271-dbc4-4c05-a07e-2f7473e5ec95 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:31.495897 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb3e01-24b8-4ab3-bc3d-f58a33379a45 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:31.496001 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7238271-dbc4-4c05-a07e-2f7473e5ec95 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:31.496191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7238271-dbc4-4c05-a07e-2f7473e5ec95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:31.496711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb3e01-24b8-4ab3-bc3d-f58a33379a45 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:31.497194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb3e01-24b8-4ab3-bc3d-f58a33379a45 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:31.498198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb3e01-24b8-4ab3-bc3d-f58a33379a45 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:31.498695 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb3e01-24b8-4ab3-bc3d-f58a33379a45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:31.500390 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb3e01-24b8-4ab3-bc3d-f58a33379a45 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:31.516951 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 56/109] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:31.522245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb3e01-24b8-4ab3-bc3d-f58a33379a45 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:31.540543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-a7238271-dbc4-4c05-a07e-2f7473e5ec95 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:31.541781 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-a7238271-dbc4-4c05-a07e-2f7473e5ec95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:31.570022 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:31.637922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb3e01-24b8-4ab3-bc3d-f58a33379a45 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:31.676570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadb8e7-dfe2-48fc-abfa-a70e1e6f7e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:31.679412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadb8e7-dfe2-48fc-abfa-a70e1e6f7e1d None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:31.679910 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadb8e7-dfe2-48fc-abfa-a70e1e6f7e1d None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:31.680382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadb8e7-dfe2-48fc-abfa-a70e1e6f7e1d None admin] PATH_INFO: `/v3/users/b3f43b29442d486daec5c2b648aa8069` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:31.696720 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 54/110] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:31.698055 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbd03c-0ee3-46ec-bd9b-37b215e4af81 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:31.698347 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbd03c-0ee3-46ec-bd9b-37b215e4af81 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:31.698627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbd03c-0ee3-46ec-bd9b-37b215e4af81 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:31.699815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aadb8e7-dfe2-48fc-abfa-a70e1e6f7e1d None admin] RBAC: Authorizing `identity:get_user(user_id=b3f43b29442d486daec5c2b648aa8069)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:31.700934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 55/111] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:31.702051 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aadb8e7-dfe2-48fc-abfa-a70e1e6f7e1d None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:31.704037 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 57/112] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:31 2022] GET /identity/v3/users/b3f43b29442d486daec5c2b648aa8069 => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:31.706473 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ea3f-b286-4c87-88db-fd2f1b23affa None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:31.706765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ea3f-b286-4c87-88db-fd2f1b23affa None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:31.707058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ea3f-b286-4c87-88db-fd2f1b23affa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:31.710716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadb8e7-dfe2-48fc-abfa-a70e1e6f7e1d None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:31.727016 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-9609ea3f-b286-4c87-88db-fd2f1b23affa None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:31.728249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-9609ea3f-b286-4c87-88db-fd2f1b23affa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:31.759242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:31.830596 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadb8e7-dfe2-48fc-abfa-a70e1e6f7e1d None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:31.835189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:31.872701 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0810f-2c57-4a6c-b314-ede1b0894cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:31.873723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0810f-2c57-4a6c-b314-ede1b0894cc2 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:31.874656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0810f-2c57-4a6c-b314-ede1b0894cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:31.875144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0810f-2c57-4a6c-b314-ede1b0894cc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:31.914526 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 56/113] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:31.915429 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609ea3f-b286-4c87-88db-fd2f1b23affa None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:31.953723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0810f-2c57-4a6c-b314-ede1b0894cc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:31.955787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0810f-2c57-4a6c-b314-ede1b0894cc2 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:32.064705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:32.070103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 58/114] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:31:32.071240 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0810f-2c57-4a6c-b314-ede1b0894cc2 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:32.120464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609ea3f-b286-4c87-88db-fd2f1b23affa None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:32.177787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0810f-2c57-4a6c-b314-ede1b0894cc2 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:32.215054 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e05d0-4ca1-40b7-bb60-e49df80683fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:32.216279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e05d0-4ca1-40b7-bb60-e49df80683fb None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:32.216781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e05d0-4ca1-40b7-bb60-e49df80683fb None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:32.217247 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e05d0-4ca1-40b7-bb60-e49df80683fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:32.228889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deefaca-d042-47e1-8506-732e20a800b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:32.229611 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deefaca-d042-47e1-8506-732e20a800b0 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:32.229900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deefaca-d042-47e1-8506-732e20a800b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:32.230196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deefaca-d042-47e1-8506-732e20a800b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:32.231040 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e05d0-4ca1-40b7-bb60-e49df80683fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:32.236111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deefaca-d042-47e1-8506-732e20a800b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:32.237616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deefaca-d042-47e1-8506-732e20a800b0 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:32.239546 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 57/115] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:31:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:32.240136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deefaca-d042-47e1-8506-732e20a800b0 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:32.240610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-d57e05d0-4ca1-40b7-bb60-e49df80683fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:31:32.240610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:32.240610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:32.240610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:32.240610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:32.240610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:32.240610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:32.240610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:32.240610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:32.240610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:32.240610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:32.240610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:32.240610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:32.240610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:32.240610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:32.240610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:32.241727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:32.241727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:32.241727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:32.241727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:32.241727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:32.241727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:32.241727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:32.241727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:32.241727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:32.241727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:32.241727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:32.241727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:32.241727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:32.241727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:32.241727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:32.241727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:32.242473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:32.242473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:32.242473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:32.242473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:32.242473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:32.242473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:32.242473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:32.242473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:32.242473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:32.242473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:32.242473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:31:32.242473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:31:32.245362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 59/116] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:31:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:32.246449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e05d0-4ca1-40b7-bb60-e49df80683fb None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:32.293801 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:32.308305 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:32.358205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e05d0-4ca1-40b7-bb60-e49df80683fb None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:32.387152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deefaca-d042-47e1-8506-732e20a800b0 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:32.399609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c943f-e802-4ce8-b50f-79ffdea89be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:32.400858 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c943f-e802-4ce8-b50f-79ffdea89be7 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:32.401352 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c943f-e802-4ce8-b50f-79ffdea89be7 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:32.401824 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c943f-e802-4ce8-b50f-79ffdea89be7 None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:32.403024 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c943f-e802-4ce8-b50f-79ffdea89be7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:32.405594 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c943f-e802-4ce8-b50f-79ffdea89be7 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:32.425478 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21aee3-79f4-4bfd-acae-8f477f069141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:32.426350 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21aee3-79f4-4bfd-acae-8f477f069141 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:32.426638 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21aee3-79f4-4bfd-acae-8f477f069141 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:32.426924 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21aee3-79f4-4bfd-acae-8f477f069141 None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:32.428389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21aee3-79f4-4bfd-acae-8f477f069141 None admin] RBAC: Authorizing `identity:get_project(project_id=e263f641bede4564a5855da9ec8514ed)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:32.429978 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21aee3-79f4-4bfd-acae-8f477f069141 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:32.432204 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 58/117] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:32 2022] GET /identity/v3/projects/e263f641bede4564a5855da9ec8514ed => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:32.432762 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21aee3-79f4-4bfd-acae-8f477f069141 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:32.447979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 60/118] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jun 22 11:31:32 2022] POST /identity/v3/projects => generated 302 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:31:32.450143 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e16dd-cf8a-4395-89c3-9c52f98af897 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:32.450653 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e16dd-cf8a-4395-89c3-9c52f98af897 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:32.451636 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e16dd-cf8a-4395-89c3-9c52f98af897 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:32.453918 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 61/119] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:32.455733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a8da8-98da-44b8-8200-3ed48d5ca114 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:32.456254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a8da8-98da-44b8-8200-3ed48d5ca114 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:32.457193 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a8da8-98da-44b8-8200-3ed48d5ca114 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:32.460110 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 62/120] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:32.461285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a8da8-98da-44b8-8200-3ed48d5ca114 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:32.543654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:32.561212 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21aee3-79f4-4bfd-acae-8f477f069141 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:32.589722 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a8da8-98da-44b8-8200-3ed48d5ca114 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:32.600960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff52bf2a-cb83-4457-b368-8791a7ba890b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:32.601844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52bf2a-cb83-4457-b368-8791a7ba890b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:32.602657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52bf2a-cb83-4457-b368-8791a7ba890b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:32.602657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52bf2a-cb83-4457-b368-8791a7ba890b None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:32.603491 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff52bf2a-cb83-4457-b368-8791a7ba890b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:32.603883 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff52bf2a-cb83-4457-b368-8791a7ba890b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:32.605514 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff52bf2a-cb83-4457-b368-8791a7ba890b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:32.608136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:32.615988 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 59/121] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:31:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:32.617999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69ef20-cd24-4e6e-90fc-f734128e8bed None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:32.618169 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69ef20-cd24-4e6e-90fc-f734128e8bed None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:32.618560 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69ef20-cd24-4e6e-90fc-f734128e8bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:32.636671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-0a69ef20-cd24-4e6e-90fc-f734128e8bed None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:32.637498 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f46649-64b3-4cd3-a247-5cead097a9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:32.638064 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-0a69ef20-cd24-4e6e-90fc-f734128e8bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:32.638546 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f46649-64b3-4cd3-a247-5cead097a9da None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:32.639122 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f46649-64b3-4cd3-a247-5cead097a9da None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:32.639598 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f46649-64b3-4cd3-a247-5cead097a9da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:32.641000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f46649-64b3-4cd3-a247-5cead097a9da None admin] RBAC: Adding query filter params (role.id=2b63bb896abc4a22ba92b589f8ba2a73, scope.project.id=e263f641bede4564a5855da9ec8514ed, user.id=b3f43b29442d486daec5c2b648aa8069) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:32.641587 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f46649-64b3-4cd3-a247-5cead097a9da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:32.643250 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f46649-64b3-4cd3-a247-5cead097a9da None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:32.651855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 63/122] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:31:32 2022] GET /identity/v3/role_assignments?user.id=b3f43b29442d486daec5c2b648aa8069&scope.project.id=e263f641bede4564a5855da9ec8514ed&role.id=2b63bb896abc4a22ba92b589f8ba2a73 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:32.651855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c9ad9-9ffe-469e-bd17-50544fef2c01 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:32.651855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c9ad9-9ffe-469e-bd17-50544fef2c01 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:32.652155 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c9ad9-9ffe-469e-bd17-50544fef2c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:32.669965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-413c9ad9-9ffe-469e-bd17-50544fef2c01 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:32.671456 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-413c9ad9-9ffe-469e-bd17-50544fef2c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:32.724338 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 60/123] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:32.725233 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69ef20-cd24-4e6e-90fc-f734128e8bed None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:32.756744 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 64/124] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:32.767814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c9ad9-9ffe-469e-bd17-50544fef2c01 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:32.867856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:32.889081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69ef20-cd24-4e6e-90fc-f734128e8bed None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:32.933152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f5bdd-18f8-4c80-ad7f-757da03f4b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:32.934058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f5bdd-18f8-4c80-ad7f-757da03f4b06 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:32.934350 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f5bdd-18f8-4c80-ad7f-757da03f4b06 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:32.934633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f5bdd-18f8-4c80-ad7f-757da03f4b06 None admin] PATH_INFO: `/v3/users/d6d9a2f332cf40a6b5bc0d534f1eab4b` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:32.935987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f5bdd-18f8-4c80-ad7f-757da03f4b06 None admin] RBAC: Authorizing `identity:get_user(user_id=d6d9a2f332cf40a6b5bc0d534f1eab4b)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:32.937949 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f5bdd-18f8-4c80-ad7f-757da03f4b06 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:32.940013 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 61/125] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:32 2022] GET /identity/v3/users/d6d9a2f332cf40a6b5bc0d534f1eab4b => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:32.944486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f5bdd-18f8-4c80-ad7f-757da03f4b06 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:32.947597 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c9ad9-9ffe-469e-bd17-50544fef2c01 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:32.956603 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:33.018308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c5dfe-cfb9-47b7-91ec-77561f0f6af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:33.019374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c5dfe-cfb9-47b7-91ec-77561f0f6af7 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:33.019665 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c5dfe-cfb9-47b7-91ec-77561f0f6af7 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:33.019943 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c5dfe-cfb9-47b7-91ec-77561f0f6af7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:33.045033 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c5dfe-cfb9-47b7-91ec-77561f0f6af7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:33.055109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-bd0c5dfe-cfb9-47b7-91ec-77561f0f6af7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:31:33.055109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:33.055109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:33.055109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:33.055109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:33.055109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:33.055109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:33.055109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:33.055109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:33.055109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:33.055109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:33.055109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:33.055109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:33.055109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:33.055109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:33.055109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:33.056020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:33.056020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:33.056020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:33.056020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:33.056020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:33.056020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:33.056020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:33.056020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:33.056020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:33.056020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:33.056020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:33.056020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:33.056020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:33.056020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:33.056020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:33.056020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:33.056759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:33.056759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:33.056759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:33.056759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:33.056759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:33.056759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:33.056759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:33.056759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:33.056759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:33.056759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:33.056759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:31:33.056759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:31:33.056759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 65/126] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:31:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:31:33.058622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c5dfe-cfb9-47b7-91ec-77561f0f6af7 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:33.117424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f5bdd-18f8-4c80-ad7f-757da03f4b06 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:33.158199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e510c-be40-4876-999c-1c84e570ee1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:33.159067 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e510c-be40-4876-999c-1c84e570ee1e None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:33.159374 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e510c-be40-4876-999c-1c84e570ee1e None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:33.159733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e510c-be40-4876-999c-1c84e570ee1e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:33.174672 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26e510c-be40-4876-999c-1c84e570ee1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:33.185402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-d26e510c-be40-4876-999c-1c84e570ee1e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:33.185402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:33.185402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:33.185402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:33.185402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:33.185402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:33.185402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:33.185402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:33.185402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:33.185402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:33.185402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:33.185402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:33.185402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:33.185402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:33.185402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:33.185402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:33.186339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:33.186339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:33.186339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:33.186339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:33.186339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:33.186339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:33.186339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:33.186339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:33.186339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:33.186339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:33.186339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:33.186339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:33.186339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:33.186339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:33.186339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:33.186339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:33.187200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:33.187200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:33.187200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:33.187200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:33.187200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:33.187200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:33.187200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:33.187200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:33.187200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:33.187200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:33.187200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:33.187200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:33.187891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 62/127] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:33.188731 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e510c-be40-4876-999c-1c84e570ee1e None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:33.211192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:33.229060 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c5dfe-cfb9-47b7-91ec-77561f0f6af7 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:33.294055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc475f8-1295-4830-9fef-8cf635992703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:33.294844 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc475f8-1295-4830-9fef-8cf635992703 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:33.295135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc475f8-1295-4830-9fef-8cf635992703 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:33.295449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc475f8-1295-4830-9fef-8cf635992703 None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:33.297593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc475f8-1295-4830-9fef-8cf635992703 None admin] RBAC: Authorizing `identity:get_project(project_id=e263f641bede4564a5855da9ec8514ed)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:33.299057 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc475f8-1295-4830-9fef-8cf635992703 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:33.303278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 66/128] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:33 2022] GET /identity/v3/projects/e263f641bede4564a5855da9ec8514ed => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:33.304128 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc475f8-1295-4830-9fef-8cf635992703 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:33.367539 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:33.391894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e510c-be40-4876-999c-1c84e570ee1e None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:33.461632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdfcbf-b5f6-48b1-a0ba-9773b1e8dda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:33.462378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdfcbf-b5f6-48b1-a0ba-9773b1e8dda7 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:33.472297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdfcbf-b5f6-48b1-a0ba-9773b1e8dda7 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:33.472590 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdfcbf-b5f6-48b1-a0ba-9773b1e8dda7 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:33.473422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fdfcbf-b5f6-48b1-a0ba-9773b1e8dda7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:33.473744 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fdfcbf-b5f6-48b1-a0ba-9773b1e8dda7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:33.478511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fdfcbf-b5f6-48b1-a0ba-9773b1e8dda7 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:33.498428 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 63/129] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:33.498960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdfcbf-b5f6-48b1-a0ba-9773b1e8dda7 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:33.506718 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc475f8-1295-4830-9fef-8cf635992703 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:33.574724 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff53ae9-9d6c-452a-85fc-b303914462c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:33.575321 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff53ae9-9d6c-452a-85fc-b303914462c2 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:33.575625 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff53ae9-9d6c-452a-85fc-b303914462c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:33.575903 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff53ae9-9d6c-452a-85fc-b303914462c2 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:33.577996 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff53ae9-9d6c-452a-85fc-b303914462c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:33.577996 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff53ae9-9d6c-452a-85fc-b303914462c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:33.578487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff53ae9-9d6c-452a-85fc-b303914462c2 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:33.596050 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 67/130] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:33.599676 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff53ae9-9d6c-452a-85fc-b303914462c2 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:33.625195 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:33.685631 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdfcbf-b5f6-48b1-a0ba-9773b1e8dda7 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:33.749662 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b622903-3e94-4ce7-b98e-f654c2f64060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:33.750570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b622903-3e94-4ce7-b98e-f654c2f64060 None admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:33.750860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b622903-3e94-4ce7-b98e-f654c2f64060 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:33.751145 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b622903-3e94-4ce7-b98e-f654c2f64060 None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed/users/d6d9a2f332cf40a6b5bc0d534f1eab4b/roles/b4be7021e4104740b8b0b3b40ecd3d59` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:33.754198 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b622903-3e94-4ce7-b98e-f654c2f64060 None admin] RBAC: Authorizing `identity:create_grant(project_id=e263f641bede4564a5855da9ec8514ed, user_id=d6d9a2f332cf40a6b5bc0d534f1eab4b, role_id=b4be7021e4104740b8b0b3b40ecd3d59)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:33.755707 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b622903-3e94-4ce7-b98e-f654c2f64060 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:33.756838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:33.776054 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 64/131] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:31:33 2022] PUT /identity/v3/projects/e263f641bede4564a5855da9ec8514ed/users/d6d9a2f332cf40a6b5bc0d534f1eab4b/roles/b4be7021e4104740b8b0b3b40ecd3d59 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:31:33.777156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b622903-3e94-4ce7-b98e-f654c2f64060 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:33.777673 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff53ae9-9d6c-452a-85fc-b303914462c2 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:33.871080 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e39b48-25c3-4d73-9135-20674f3f252b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:33.871929 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e39b48-25c3-4d73-9135-20674f3f252b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:33.872217 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e39b48-25c3-4d73-9135-20674f3f252b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:33.872501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e39b48-25c3-4d73-9135-20674f3f252b None admin] PATH_INFO: `/v3/users/8441761690594b9d8a423147d5d6d62d` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:33.873816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e39b48-25c3-4d73-9135-20674f3f252b None admin] RBAC: Authorizing `identity:get_user(user_id=8441761690594b9d8a423147d5d6d62d)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:33.875326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e39b48-25c3-4d73-9135-20674f3f252b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:33.877100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 68/132] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:33 2022] GET /identity/v3/users/8441761690594b9d8a423147d5d6d62d => generated 247 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:33.882851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e39b48-25c3-4d73-9135-20674f3f252b None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:33.915258 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b622903-3e94-4ce7-b98e-f654c2f64060 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:33.984977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:34.003380 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05952c9-1042-44c9-af45-d4b64ef00be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:34.004773 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05952c9-1042-44c9-af45-d4b64ef00be8 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:34.006052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05952c9-1042-44c9-af45-d4b64ef00be8 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:34.006665 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05952c9-1042-44c9-af45-d4b64ef00be8 None admin] PATH_INFO: `/v3/users/d6d9a2f332cf40a6b5bc0d534f1eab4b` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:34.010242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05952c9-1042-44c9-af45-d4b64ef00be8 None admin] RBAC: Authorizing `identity:get_user(user_id=d6d9a2f332cf40a6b5bc0d534f1eab4b)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:34.012889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05952c9-1042-44c9-af45-d4b64ef00be8 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:34.015630 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 65/133] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:33 2022] GET /identity/v3/users/d6d9a2f332cf40a6b5bc0d534f1eab4b => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:34.024932 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05952c9-1042-44c9-af45-d4b64ef00be8 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:34.125245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e39b48-25c3-4d73-9135-20674f3f252b None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:34.173553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d72047-f606-4f91-aa59-e6b2b0ccdc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:34.175203 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d72047-f606-4f91-aa59-e6b2b0ccdc3f None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:34.175754 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d72047-f606-4f91-aa59-e6b2b0ccdc3f None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:34.176243 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d72047-f606-4f91-aa59-e6b2b0ccdc3f None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:34.179855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d72047-f606-4f91-aa59-e6b2b0ccdc3f None admin] RBAC: Authorizing `identity:get_project(project_id=e263f641bede4564a5855da9ec8514ed)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:34.181715 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d72047-f606-4f91-aa59-e6b2b0ccdc3f None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:34.184040 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 69/134] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:33 2022] GET /identity/v3/projects/e263f641bede4564a5855da9ec8514ed => generated 298 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:34.196677 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d72047-f606-4f91-aa59-e6b2b0ccdc3f None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:34.233870 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:34.313313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d72047-f606-4f91-aa59-e6b2b0ccdc3f None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:34.333694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05952c9-1042-44c9-af45-d4b64ef00be8 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:34.356723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cbbceb-54fe-4183-a0e5-d105078151c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:34.358006 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbbceb-54fe-4183-a0e5-d105078151c0 None admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:34.358494 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbbceb-54fe-4183-a0e5-d105078151c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:34.359245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbbceb-54fe-4183-a0e5-d105078151c0 None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed/users/8441761690594b9d8a423147d5d6d62d/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:34.361649 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cbbceb-54fe-4183-a0e5-d105078151c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e263f641bede4564a5855da9ec8514ed, user_id=8441761690594b9d8a423147d5d6d62d, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:34.363338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cbbceb-54fe-4183-a0e5-d105078151c0 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:34.381542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 70/135] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:31:34 2022] PUT /identity/v3/projects/e263f641bede4564a5855da9ec8514ed/users/8441761690594b9d8a423147d5d6d62d/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:31:34.383456 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f019ace-ac22-4889-afff-6aca7fe98277 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:34.383951 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f019ace-ac22-4889-afff-6aca7fe98277 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:34.384534 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f019ace-ac22-4889-afff-6aca7fe98277 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:34.386241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 71/136] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:34.397286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab64b3-cf54-49b7-be10-5730ddf69bb5 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:34.398136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab64b3-cf54-49b7-be10-5730ddf69bb5 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:34.398637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab64b3-cf54-49b7-be10-5730ddf69bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:34.433417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-d7ab64b3-cf54-49b7-be10-5730ddf69bb5 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:34.435269 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d7ab64b3-cf54-49b7-be10-5730ddf69bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:34.456193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbeab21-e2db-46da-a0c8-66f9c7ddf9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:34.457195 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbeab21-e2db-46da-a0c8-66f9c7ddf9f0 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:34.457483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbeab21-e2db-46da-a0c8-66f9c7ddf9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:34.457763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbeab21-e2db-46da-a0c8-66f9c7ddf9f0 None admin] PATH_INFO: `/v3/projects/0ce6633b03a947ecb728e95eea7a60b5` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:34.464445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbeab21-e2db-46da-a0c8-66f9c7ddf9f0 None admin] RBAC: Authorizing `identity:get_project(project_id=0ce6633b03a947ecb728e95eea7a60b5)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:34.465942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbeab21-e2db-46da-a0c8-66f9c7ddf9f0 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:34.468157 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 66/137] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:34 2022] GET /identity/v3/projects/0ce6633b03a947ecb728e95eea7a60b5 => generated 312 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:34.476419 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbeab21-e2db-46da-a0c8-66f9c7ddf9f0 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:34.557484 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 72/138] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 22 11:31:34.626084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab64b3-cf54-49b7-be10-5730ddf69bb5 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:34.669245 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:34.700924 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:34.745001 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbeab21-e2db-46da-a0c8-66f9c7ddf9f0 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:34.801264 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f787b1-a756-4fa6-a1c9-6a87c67bdd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:34.802838 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f787b1-a756-4fa6-a1c9-6a87c67bdd2c None admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:34.803128 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f787b1-a756-4fa6-a1c9-6a87c67bdd2c None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:34.803431 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f787b1-a756-4fa6-a1c9-6a87c67bdd2c None admin] PATH_INFO: `/v3/projects/0ce6633b03a947ecb728e95eea7a60b5/users/d6d9a2f332cf40a6b5bc0d534f1eab4b/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:34.809379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f787b1-a756-4fa6-a1c9-6a87c67bdd2c None admin] RBAC: Authorizing `identity:create_grant(project_id=0ce6633b03a947ecb728e95eea7a60b5, user_id=d6d9a2f332cf40a6b5bc0d534f1eab4b, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:34.811198 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f787b1-a756-4fa6-a1c9-6a87c67bdd2c None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:34.826373 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab64b3-cf54-49b7-be10-5730ddf69bb5 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:34.833574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 67/139] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:31:34 2022] PUT /identity/v3/projects/0ce6633b03a947ecb728e95eea7a60b5/users/d6d9a2f332cf40a6b5bc0d534f1eab4b/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:31:34.884286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08d7b4-6153-4ea0-81e1-b818675f09ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:34.885136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08d7b4-6153-4ea0-81e1-b818675f09ce None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:34.885422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08d7b4-6153-4ea0-81e1-b818675f09ce None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:34.885705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08d7b4-6153-4ea0-81e1-b818675f09ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:34.899422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08d7b4-6153-4ea0-81e1-b818675f09ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:34.907508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-6c08d7b4-6153-4ea0-81e1-b818675f09ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:34.907508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:34.907508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:34.907508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:34.907508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:34.907508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:34.907508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:34.907508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:34.907508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:34.907508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:34.907508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:34.907508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:34.907508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:34.907508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:34.907508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:34.907508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:34.908459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:34.908459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:34.908459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:34.908459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:34.908459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:34.908459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:34.908459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:34.908459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:34.908459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:34.908459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:34.908459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:34.908459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:34.908459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:34.908459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:34.908459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:34.908459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:34.909248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:34.909248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:34.909248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:34.909248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:34.909248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:34.909248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:34.909248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:34.909248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:34.909248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:34.909248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:34.909248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:34.909248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:31:34.909248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 73/140] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:31:34.915239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f787b1-a756-4fa6-a1c9-6a87c67bdd2c None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:35.030458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f787b1-a756-4fa6-a1c9-6a87c67bdd2c None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:35.056276 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:35.067759 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d338e-72e9-4f26-937b-32c69adfe066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:35.068465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d338e-72e9-4f26-937b-32c69adfe066 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:35.069173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d338e-72e9-4f26-937b-32c69adfe066 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:35.069463 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d338e-72e9-4f26-937b-32c69adfe066 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:35.070265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d338e-72e9-4f26-937b-32c69adfe066 None admin] RBAC: Adding query filter params (name=member) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:35.070960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d338e-72e9-4f26-937b-32c69adfe066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:35.072862 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d338e-72e9-4f26-937b-32c69adfe066 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:35.087562 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 68/141] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:35.089955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08d7b4-6153-4ea0-81e1-b818675f09ce None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:35.200311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:35.222195 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08d7b4-6153-4ea0-81e1-b818675f09ce None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:35.259977 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8dd37-89d2-48eb-9606-b3155821f9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:35.260866 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8dd37-89d2-48eb-9606-b3155821f9a1 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:35.261157 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8dd37-89d2-48eb-9606-b3155821f9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:35.261441 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8dd37-89d2-48eb-9606-b3155821f9a1 None admin] PATH_INFO: `/v3/users/d6d9a2f332cf40a6b5bc0d534f1eab4b` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:35.262665 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8dd37-89d2-48eb-9606-b3155821f9a1 None admin] RBAC: Authorizing `identity:get_user(user_id=d6d9a2f332cf40a6b5bc0d534f1eab4b)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:35.264197 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8dd37-89d2-48eb-9606-b3155821f9a1 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:35.266218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 74/142] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:35 2022] GET /identity/v3/users/d6d9a2f332cf40a6b5bc0d534f1eab4b => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:35.272496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d338e-72e9-4f26-937b-32c69adfe066 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:35.283886 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d3c4c-9fc3-4e99-99a0-520e665b02a8 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:35.287258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d3c4c-9fc3-4e99-99a0-520e665b02a8 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:35.287591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d3c4c-9fc3-4e99-99a0-520e665b02a8 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:35.289174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 75/143] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:35.294942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1630a83-2492-4fe8-8a01-dc6e57a5c96e None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:35.295234 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1630a83-2492-4fe8-8a01-dc6e57a5c96e None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:35.295561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1630a83-2492-4fe8-8a01-dc6e57a5c96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:35.327519 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-b1630a83-2492-4fe8-8a01-dc6e57a5c96e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:35.328930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b1630a83-2492-4fe8-8a01-dc6e57a5c96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:35.467558 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 76/144] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:35.472209 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711d9e4-d962-42fd-8898-ec6dc86f3cd6 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:35.472508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711d9e4-d962-42fd-8898-ec6dc86f3cd6 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:35.472795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711d9e4-d962-42fd-8898-ec6dc86f3cd6 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:35.474170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 77/145] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:35.476476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d338e-72e9-4f26-937b-32c69adfe066 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:35.480151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d8364-7dce-4c5e-bfda-65ee8554c4bd None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:35.480437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d8364-7dce-4c5e-bfda-65ee8554c4bd None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:35.480715 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d8364-7dce-4c5e-bfda-65ee8554c4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:35.521863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-f11d8364-7dce-4c5e-bfda-65ee8554c4bd None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:35.525154 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f11d8364-7dce-4c5e-bfda-65ee8554c4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:35.554252 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df9ccb-5307-447e-83f1-824a24fddd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:35.554922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df9ccb-5307-447e-83f1-824a24fddd2b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:35.555224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df9ccb-5307-447e-83f1-824a24fddd2b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:35.555533 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df9ccb-5307-447e-83f1-824a24fddd2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:35.621696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:35.644527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df9ccb-5307-447e-83f1-824a24fddd2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:35.646582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df9ccb-5307-447e-83f1-824a24fddd2b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:35.670813 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 78/146] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:35.672246 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d8364-7dce-4c5e-bfda-65ee8554c4bd None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:35.690459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:35.817132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 69/147] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 22 11:31:35.818539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df9ccb-5307-447e-83f1-824a24fddd2b None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:35.865418 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d8364-7dce-4c5e-bfda-65ee8554c4bd None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:35.911095 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e776ed-5d3b-41e3-a0ae-b83b81ac89da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:35.911948 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e776ed-5d3b-41e3-a0ae-b83b81ac89da None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:35.912227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e776ed-5d3b-41e3-a0ae-b83b81ac89da None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:35.919468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e776ed-5d3b-41e3-a0ae-b83b81ac89da None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:35.922825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e776ed-5d3b-41e3-a0ae-b83b81ac89da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:35.931529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-74e776ed-5d3b-41e3-a0ae-b83b81ac89da None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 22 11:31:35.931529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:35.931529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:35.931529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:35.931529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:35.931529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:35.931529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:35.931529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:35.931529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:35.931529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:35.931529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:35.931529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:35.931529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:35.931529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:35.931529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:35.931529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:35.932434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:35.932434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:35.932434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:35.932434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:35.932434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:35.932434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:35.932434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:35.932434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:35.932434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:35.932434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:35.932434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:35.932434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:35.932434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:35.932434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:35.932434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:35.932434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:35.933200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:35.933200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:35.933200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:35.933200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:35.933200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:35.933200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:35.933200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:35.933200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:35.933200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:35.933200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:35.933200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 22 11:31:35.933200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:31:35.933200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 79/148] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:35.933840 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e776ed-5d3b-41e3-a0ae-b83b81ac89da None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:35.979371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:36.007025 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df9ccb-5307-447e-83f1-824a24fddd2b None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:36.008804 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:36.047205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e776ed-5d3b-41e3-a0ae-b83b81ac89da None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:36.087041 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3d52e-773d-4f76-a66b-1354170caaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:36.087847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3d52e-773d-4f76-a66b-1354170caaa2 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:36.088125 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3d52e-773d-4f76-a66b-1354170caaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:36.088401 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3d52e-773d-4f76-a66b-1354170caaa2 None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:36.089596 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3d52e-773d-4f76-a66b-1354170caaa2 None admin] RBAC: Authorizing `identity:get_project(project_id=e263f641bede4564a5855da9ec8514ed)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:36.091063 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3d52e-773d-4f76-a66b-1354170caaa2 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:36.092796 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 80/149] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:35 2022] GET /identity/v3/projects/e263f641bede4564a5855da9ec8514ed => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:36.093561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3d52e-773d-4f76-a66b-1354170caaa2 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:36.112076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ddf3c-5c0c-452d-b878-adf7dd890c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:36.112898 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ddf3c-5c0c-452d-b878-adf7dd890c39 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:36.113187 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ddf3c-5c0c-452d-b878-adf7dd890c39 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:36.113467 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ddf3c-5c0c-452d-b878-adf7dd890c39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:36.115235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ddf3c-5c0c-452d-b878-adf7dd890c39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:36.116739 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ddf3c-5c0c-452d-b878-adf7dd890c39 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:36.118391 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 70/150] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:31:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:36.137645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ddf3c-5c0c-452d-b878-adf7dd890c39 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:36.202202 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3d52e-773d-4f76-a66b-1354170caaa2 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:36.238395 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5858326-5780-4c79-9a7c-eba830b4e426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:36.239073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5858326-5780-4c79-9a7c-eba830b4e426 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:36.239394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5858326-5780-4c79-9a7c-eba830b4e426 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:36.240211 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5858326-5780-4c79-9a7c-eba830b4e426 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:36.240964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5858326-5780-4c79-9a7c-eba830b4e426 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:36.241270 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5858326-5780-4c79-9a7c-eba830b4e426 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:36.242652 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5858326-5780-4c79-9a7c-eba830b4e426 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:36.245075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:36.253557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 81/151] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:36.254670 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5858326-5780-4c79-9a7c-eba830b4e426 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:36.343815 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ddf3c-5c0c-452d-b878-adf7dd890c39 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:36.386012 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917d51b-dcdd-4934-b1b4-e376a3cb7ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:36.386676 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917d51b-dcdd-4934-b1b4-e376a3cb7ff8 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:36.386964 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917d51b-dcdd-4934-b1b4-e376a3cb7ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:36.387237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917d51b-dcdd-4934-b1b4-e376a3cb7ff8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:36.388111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917d51b-dcdd-4934-b1b4-e376a3cb7ff8 None admin] RBAC: Adding query filter params (role.id=f2336919c5104f6a8210e99994f4d455, scope.project.id=e263f641bede4564a5855da9ec8514ed, user.id=d6d9a2f332cf40a6b5bc0d534f1eab4b) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:36.388424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917d51b-dcdd-4934-b1b4-e376a3cb7ff8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:36.390152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917d51b-dcdd-4934-b1b4-e376a3cb7ff8 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:36.396430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 71/152] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:31:36 2022] GET /identity/v3/role_assignments?user.id=d6d9a2f332cf40a6b5bc0d534f1eab4b&scope.project.id=e263f641bede4564a5855da9ec8514ed&role.id=f2336919c5104f6a8210e99994f4d455 => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:36.397223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917d51b-dcdd-4934-b1b4-e376a3cb7ff8 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:36.417730 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:36.440418 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5858326-5780-4c79-9a7c-eba830b4e426 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:36.510702 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f860f74-ee32-4a9d-941a-b0fe9d4bb9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:36.512898 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f860f74-ee32-4a9d-941a-b0fe9d4bb9a6 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:36.514459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f860f74-ee32-4a9d-941a-b0fe9d4bb9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:36.515379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f860f74-ee32-4a9d-941a-b0fe9d4bb9a6 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:36.517775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f860f74-ee32-4a9d-941a-b0fe9d4bb9a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:36.520336 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f860f74-ee32-4a9d-941a-b0fe9d4bb9a6 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:36.560006 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 82/153] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jun 22 11:31:36 2022] POST /identity/v3/users => generated 283 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:31:36.578604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917d51b-dcdd-4934-b1b4-e376a3cb7ff8 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:36.630004 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb45abe-730d-4f44-99c9-b3be14378e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:36.630854 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb45abe-730d-4f44-99c9-b3be14378e63 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:36.631138 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb45abe-730d-4f44-99c9-b3be14378e63 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:36.631547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb45abe-730d-4f44-99c9-b3be14378e63 None admin] PATH_INFO: `/v3/users/b3f43b29442d486daec5c2b648aa8069` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:36.635096 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb45abe-730d-4f44-99c9-b3be14378e63 None admin] RBAC: Authorizing `identity:get_user(user_id=b3f43b29442d486daec5c2b648aa8069)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:36.636724 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb45abe-730d-4f44-99c9-b3be14378e63 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:36.638860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 72/154] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:36 2022] GET /identity/v3/users/b3f43b29442d486daec5c2b648aa8069 => generated 289 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:36.644969 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f860f74-ee32-4a9d-941a-b0fe9d4bb9a6 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:36.731053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:36.780506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f860f74-ee32-4a9d-941a-b0fe9d4bb9a6 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:36.785061 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6dc86-08b7-4fda-ac0e-25f60086148c None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:36.785349 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6dc86-08b7-4fda-ac0e-25f60086148c None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:36.785631 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6dc86-08b7-4fda-ac0e-25f60086148c None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:36.789838 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 73/155] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:36.795027 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3109d-cd97-4b79-b812-e4580c3e963f None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:36.795313 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3109d-cd97-4b79-b812-e4580c3e963f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:36.795618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3109d-cd97-4b79-b812-e4580c3e963f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:36.818397 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ccd8b-b5d6-4d59-b9c9-1dd650f00493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:36.819654 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ccd8b-b5d6-4d59-b9c9-1dd650f00493 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:36.820173 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ccd8b-b5d6-4d59-b9c9-1dd650f00493 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:36.820657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ccd8b-b5d6-4d59-b9c9-1dd650f00493 None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:36.820946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d8c3109d-cd97-4b79-b812-e4580c3e963f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:36.822166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d8c3109d-cd97-4b79-b812-e4580c3e963f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:36.822594 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ccd8b-b5d6-4d59-b9c9-1dd650f00493 None admin] RBAC: Authorizing `identity:get_project(project_id=e263f641bede4564a5855da9ec8514ed)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:36.824296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ccd8b-b5d6-4d59-b9c9-1dd650f00493 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:36.826698 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 83/156] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:36 2022] GET /identity/v3/projects/e263f641bede4564a5855da9ec8514ed => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:36.832442 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ccd8b-b5d6-4d59-b9c9-1dd650f00493 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:36.898506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 74/157] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 22 11:31:36.924658 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3109d-cd97-4b79-b812-e4580c3e963f None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:36.949363 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:36.953214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ccd8b-b5d6-4d59-b9c9-1dd650f00493 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:36.983749 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:36.997329 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b460e-7696-433f-8b14-7204576306bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:36.998597 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b460e-7696-433f-8b14-7204576306bc None admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:36.998982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b460e-7696-433f-8b14-7204576306bc None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:36.999377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b460e-7696-433f-8b14-7204576306bc None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed/users/b3f43b29442d486daec5c2b648aa8069/roles/2b63bb896abc4a22ba92b589f8ba2a73` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:37.001660 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b460e-7696-433f-8b14-7204576306bc None admin] RBAC: Authorizing `identity:create_grant(project_id=e263f641bede4564a5855da9ec8514ed, user_id=b3f43b29442d486daec5c2b648aa8069, role_id=2b63bb896abc4a22ba92b589f8ba2a73)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:37.003431 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b460e-7696-433f-8b14-7204576306bc None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:37.023890 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 84/158] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:31:36 2022] PUT /identity/v3/projects/e263f641bede4564a5855da9ec8514ed/users/b3f43b29442d486daec5c2b648aa8069/roles/2b63bb896abc4a22ba92b589f8ba2a73 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:31:37.045616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3109d-cd97-4b79-b812-e4580c3e963f None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:37.085976 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241910c7-0ac5-404f-8b26-9a38e94a56fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:37.086811 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241910c7-0ac5-404f-8b26-9a38e94a56fe None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:37.087091 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241910c7-0ac5-404f-8b26-9a38e94a56fe None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:37.087415 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241910c7-0ac5-404f-8b26-9a38e94a56fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:37.099966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241910c7-0ac5-404f-8b26-9a38e94a56fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:37.108587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-241910c7-0ac5-404f-8b26-9a38e94a56fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:31:37.108587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:37.108587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:37.108587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:37.108587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:37.108587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:37.108587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:37.108587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:37.108587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:37.108587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:37.108587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:37.108587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:37.108587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:37.108587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:37.108587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:37.108587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:37.109525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:37.109525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:37.109525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:37.109525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:37.109525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:37.109525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:37.109525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:37.109525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:37.109525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:37.109525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:37.109525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:37.109525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:37.109525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:37.109525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:37.109525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:37.109525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:37.110335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:37.110335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:37.110335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:37.110335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:37.110335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:37.110335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:37.110335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:37.110335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:37.110335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:37.110335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:37.110335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:31:37.110335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:37.110335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 75/159] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:31:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:37.120407 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b460e-7696-433f-8b14-7204576306bc None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:37.240319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b460e-7696-433f-8b14-7204576306bc None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:37.280653 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acf1e0-c948-4386-b43c-7f9822e5e52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:37.281327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acf1e0-c948-4386-b43c-7f9822e5e52b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:37.281608 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acf1e0-c948-4386-b43c-7f9822e5e52b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:37.281936 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acf1e0-c948-4386-b43c-7f9822e5e52b None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:37.282621 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55acf1e0-c948-4386-b43c-7f9822e5e52b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:37.282928 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55acf1e0-c948-4386-b43c-7f9822e5e52b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:37.284409 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55acf1e0-c948-4386-b43c-7f9822e5e52b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:37.293456 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 85/160] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:31:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:37.298800 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241910c7-0ac5-404f-8b26-9a38e94a56fe None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:37.384171 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:37.409027 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241910c7-0ac5-404f-8b26-9a38e94a56fe None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:37.447102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85feb90f-b23e-45e9-89ed-8160d206ebcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:37.448025 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85feb90f-b23e-45e9-89ed-8160d206ebcd None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:37.448307 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85feb90f-b23e-45e9-89ed-8160d206ebcd None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:37.448587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85feb90f-b23e-45e9-89ed-8160d206ebcd None admin] PATH_INFO: `/v3/users/d6d9a2f332cf40a6b5bc0d534f1eab4b` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:37.449817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85feb90f-b23e-45e9-89ed-8160d206ebcd None admin] RBAC: Authorizing `identity:get_user(user_id=d6d9a2f332cf40a6b5bc0d534f1eab4b)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:37.452154 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192c793-683a-4231-a576-4492bc52abab None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:37.452617 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192c793-683a-4231-a576-4492bc52abab None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:37.452700 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192c793-683a-4231-a576-4492bc52abab None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:37.454238 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 86/161] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:31:37.454897 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85feb90f-b23e-45e9-89ed-8160d206ebcd None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:37.456677 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 76/162] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:37 2022] GET /identity/v3/users/d6d9a2f332cf40a6b5bc0d534f1eab4b => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:37.460323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f99aa-0564-4f78-9a22-04c0254cd620 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:37.460726 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f99aa-0564-4f78-9a22-04c0254cd620 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:37.461107 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f99aa-0564-4f78-9a22-04c0254cd620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:37.462890 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85feb90f-b23e-45e9-89ed-8160d206ebcd None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:37.485501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-3e5f99aa-0564-4f78-9a22-04c0254cd620 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:37.487055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-3e5f99aa-0564-4f78-9a22-04c0254cd620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:37.581466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:37.598690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 87/163] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:37.631121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f99aa-0564-4f78-9a22-04c0254cd620 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:37.656577 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85feb90f-b23e-45e9-89ed-8160d206ebcd None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:37.705637 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a2c67-f540-46b5-8379-0bf258e15762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:37.706416 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a2c67-f540-46b5-8379-0bf258e15762 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:37.706762 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a2c67-f540-46b5-8379-0bf258e15762 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:37.707111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a2c67-f540-46b5-8379-0bf258e15762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:37.749355 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:37.763109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f99aa-0564-4f78-9a22-04c0254cd620 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:37.789326 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a2c67-f540-46b5-8379-0bf258e15762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:37.790934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a2c67-f540-46b5-8379-0bf258e15762 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:37.800958 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d0e6a-c280-4912-b997-7fc32c72e04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:37.801767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d0e6a-c280-4912-b997-7fc32c72e04c None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:37.802610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d0e6a-c280-4912-b997-7fc32c72e04c None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:37.802920 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d0e6a-c280-4912-b997-7fc32c72e04c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:37.815185 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d0e6a-c280-4912-b997-7fc32c72e04c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:37.826136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-c41d0e6a-c280-4912-b997-7fc32c72e04c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:31:37.826136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:37.826136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:37.826136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:37.826136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:37.826136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:37.826136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:37.826136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:37.826136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:37.826136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:37.826136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:37.826136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:37.826136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:37.826136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:37.826136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:37.826136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:37.827061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:37.827061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:37.827061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:37.827061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:37.827061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:37.827061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:37.827061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:37.827061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:37.827061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:37.827061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:37.827061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:37.827061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:37.827061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:37.827061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:37.827061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:37.827061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:37.828362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:37.828362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:37.828362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:37.828362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:37.828362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:37.828362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:37.828362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:37.828362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:37.828362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:37.828362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:37.828362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:31:37.828362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:31:37.828362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 88/164] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:31:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:37.829268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dc03d-c827-4497-96db-37aba01dc110 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:37.829268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dc03d-c827-4497-96db-37aba01dc110 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:37.829657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dc03d-c827-4497-96db-37aba01dc110 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:37.830814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 89/165] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:37.836760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ecd8a1-2e27-49f3-b81b-399c28d2442b None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:37.837213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ecd8a1-2e27-49f3-b81b-399c28d2442b None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:37.837505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ecd8a1-2e27-49f3-b81b-399c28d2442b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:37.855546 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c3ecd8a1-2e27-49f3-b81b-399c28d2442b None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:37.856754 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c3ecd8a1-2e27-49f3-b81b-399c28d2442b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:37.888428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:37.910626 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 77/166] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:31:37.912183 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a2c67-f540-46b5-8379-0bf258e15762 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:37.936453 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 90/167] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 22 11:31:37.940626 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ecd8a1-2e27-49f3-b81b-399c28d2442b None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:37.975030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:38.056282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a2c67-f540-46b5-8379-0bf258e15762 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:38.080379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ecd8a1-2e27-49f3-b81b-399c28d2442b None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:38.110089 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d44e1-d2c9-49c5-9d5b-a277e7b00537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:38.112656 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d44e1-d2c9-49c5-9d5b-a277e7b00537 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:38.113147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d44e1-d2c9-49c5-9d5b-a277e7b00537 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:38.113756 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d44e1-d2c9-49c5-9d5b-a277e7b00537 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:38.115645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d44e1-d2c9-49c5-9d5b-a277e7b00537 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:38.116954 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d44e1-d2c9-49c5-9d5b-a277e7b00537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:38.118629 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3d44e1-d2c9-49c5-9d5b-a277e7b00537 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:38.136222 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 78/168] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:38.141964 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d44e1-d2c9-49c5-9d5b-a277e7b00537 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:38.161117 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75875ab0-98df-4480-be0c-ccea7724abc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:38.162555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75875ab0-98df-4480-be0c-ccea7724abc3 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:38.162835 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75875ab0-98df-4480-be0c-ccea7724abc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:38.163110 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75875ab0-98df-4480-be0c-ccea7724abc3 None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:38.168231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75875ab0-98df-4480-be0c-ccea7724abc3 None admin] RBAC: Authorizing `identity:get_project(project_id=e263f641bede4564a5855da9ec8514ed)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:38.170908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75875ab0-98df-4480-be0c-ccea7724abc3 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:38.171610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 91/169] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:37 2022] GET /identity/v3/projects/e263f641bede4564a5855da9ec8514ed => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:38.172389 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75875ab0-98df-4480-be0c-ccea7724abc3 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:38.224400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:38.279220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d44e1-d2c9-49c5-9d5b-a277e7b00537 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:38.287628 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:38.328087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290323e-7a5a-4f6b-974f-0f3c413263fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:38.329291 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290323e-7a5a-4f6b-974f-0f3c413263fb None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:38.329800 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290323e-7a5a-4f6b-974f-0f3c413263fb None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:38.330290 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290323e-7a5a-4f6b-974f-0f3c413263fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:38.349657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9290323e-7a5a-4f6b-974f-0f3c413263fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:38.366180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-9290323e-7a5a-4f6b-974f-0f3c413263fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:38.366180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:38.366180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:38.366180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:38.366180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:38.366180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:38.366180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:38.366180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:38.366180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:38.366180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:38.366180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:38.366180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:38.366180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:38.366180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:38.366180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:38.366180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:38.367077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:38.367077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:38.367077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:38.367077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:38.367077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:38.367077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:38.367077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:38.367077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:38.367077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:38.367077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:38.367077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:38.367077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:38.367077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:38.367077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:38.367077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:38.367077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:38.367888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:38.367888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:38.367888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:38.367888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:38.367888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:38.367888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:38.367888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:38.367888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:38.367888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:38.367888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:38.367888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:38.367888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:38.370326 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 79/170] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:38.373022 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290323e-7a5a-4f6b-974f-0f3c413263fb None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:38.378201 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75875ab0-98df-4480-be0c-ccea7724abc3 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:38.472088 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630caf0-c817-4028-8129-d380885e16c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:38.472977 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630caf0-c817-4028-8129-d380885e16c3 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:38.473259 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630caf0-c817-4028-8129-d380885e16c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:38.473531 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630caf0-c817-4028-8129-d380885e16c3 None admin] PATH_INFO: `/v3/users/8441761690594b9d8a423147d5d6d62d` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:38.474893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630caf0-c817-4028-8129-d380885e16c3 None admin] RBAC: Authorizing `identity:get_user(user_id=8441761690594b9d8a423147d5d6d62d)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:38.479787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630caf0-c817-4028-8129-d380885e16c3 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:38.481542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 92/171] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:38 2022] GET /identity/v3/users/8441761690594b9d8a423147d5d6d62d => generated 247 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:38.486099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630caf0-c817-4028-8129-d380885e16c3 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:38.493193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290323e-7a5a-4f6b-974f-0f3c413263fb None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:38.542372 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076dbe27-1de0-40ad-b14a-eec5cadda3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:38.543497 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076dbe27-1de0-40ad-b14a-eec5cadda3dd None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:38.548563 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076dbe27-1de0-40ad-b14a-eec5cadda3dd None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:38.549128 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076dbe27-1de0-40ad-b14a-eec5cadda3dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:38.555370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076dbe27-1de0-40ad-b14a-eec5cadda3dd None admin] RBAC: Adding query filter params (role.id=b4be7021e4104740b8b0b3b40ecd3d59, scope.project.id=e263f641bede4564a5855da9ec8514ed, user.id=d6d9a2f332cf40a6b5bc0d534f1eab4b) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:38.555941 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076dbe27-1de0-40ad-b14a-eec5cadda3dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:38.557654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076dbe27-1de0-40ad-b14a-eec5cadda3dd None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:38.568156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 80/172] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:31:38 2022] GET /identity/v3/role_assignments?user.id=d6d9a2f332cf40a6b5bc0d534f1eab4b&scope.project.id=e263f641bede4564a5855da9ec8514ed&role.id=b4be7021e4104740b8b0b3b40ecd3d59 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:38.577792 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076dbe27-1de0-40ad-b14a-eec5cadda3dd None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:38.673854 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:38.727527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630caf0-c817-4028-8129-d380885e16c3 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:38.731698 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:38.821183 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076dbe27-1de0-40ad-b14a-eec5cadda3dd None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:38.836740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb728d7-938c-46c6-9c06-e7334b6dc42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:38.837440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb728d7-938c-46c6-9c06-e7334b6dc42c None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:38.837729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb728d7-938c-46c6-9c06-e7334b6dc42c None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:38.838013 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb728d7-938c-46c6-9c06-e7334b6dc42c None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:38.838787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb728d7-938c-46c6-9c06-e7334b6dc42c None admin] RBAC: Adding query filter params (name=member) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:38.839100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb728d7-938c-46c6-9c06-e7334b6dc42c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:38.840670 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb728d7-938c-46c6-9c06-e7334b6dc42c None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:38.857756 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55808af5-c350-43c3-8ae7-3427e6f3c9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:38.858528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 93/173] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:38.858636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55808af5-c350-43c3-8ae7-3427e6f3c9cb None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:38.858749 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55808af5-c350-43c3-8ae7-3427e6f3c9cb None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:38.858958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55808af5-c350-43c3-8ae7-3427e6f3c9cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:38.863817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb728d7-938c-46c6-9c06-e7334b6dc42c None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:38.945192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55808af5-c350-43c3-8ae7-3427e6f3c9cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:38.951383 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55808af5-c350-43c3-8ae7-3427e6f3c9cb None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:38.990294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:39.060971 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 81/174] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:31:39.069609 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55808af5-c350-43c3-8ae7-3427e6f3c9cb None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:39.197395 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb728d7-938c-46c6-9c06-e7334b6dc42c None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:39.221870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55808af5-c350-43c3-8ae7-3427e6f3c9cb None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:39.261024 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f562891-d6b9-49b1-910c-333ec532a0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:39.261902 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f562891-d6b9-49b1-910c-333ec532a0f4 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:39.262185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f562891-d6b9-49b1-910c-333ec532a0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:39.262460 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f562891-d6b9-49b1-910c-333ec532a0f4 None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:39.263368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:39.268385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f562891-d6b9-49b1-910c-333ec532a0f4 None admin] RBAC: Authorizing `identity:get_project(project_id=e263f641bede4564a5855da9ec8514ed)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:39.269946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f562891-d6b9-49b1-910c-333ec532a0f4 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:39.272218 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 82/175] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:39 2022] GET /identity/v3/projects/e263f641bede4564a5855da9ec8514ed => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:39.278580 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f562891-d6b9-49b1-910c-333ec532a0f4 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:39.328070 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09029916-ecc0-4185-bd78-d1521cd10859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:39.328962 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09029916-ecc0-4185-bd78-d1521cd10859 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:39.329241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09029916-ecc0-4185-bd78-d1521cd10859 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:39.329520 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09029916-ecc0-4185-bd78-d1521cd10859 None admin] PATH_INFO: `/v3/users/d6d9a2f332cf40a6b5bc0d534f1eab4b` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:39.336099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09029916-ecc0-4185-bd78-d1521cd10859 None admin] RBAC: Authorizing `identity:get_user(user_id=d6d9a2f332cf40a6b5bc0d534f1eab4b)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:39.337739 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09029916-ecc0-4185-bd78-d1521cd10859 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:39.339842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 94/176] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:38 2022] GET /identity/v3/users/d6d9a2f332cf40a6b5bc0d534f1eab4b => generated 275 bytes in 476 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:31:39.348345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09029916-ecc0-4185-bd78-d1521cd10859 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:39.389536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f562891-d6b9-49b1-910c-333ec532a0f4 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:39.475201 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19b6c5-9672-438f-af6a-dad40a641f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:39.475818 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19b6c5-9672-438f-af6a-dad40a641f75 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:39.476099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19b6c5-9672-438f-af6a-dad40a641f75 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:39.476485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19b6c5-9672-438f-af6a-dad40a641f75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:39.477052 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:39.477476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19b6c5-9672-438f-af6a-dad40a641f75 None admin] RBAC: Adding query filter params (role.id=96c584428d924fdba9671949f760c406, scope.project.id=e263f641bede4564a5855da9ec8514ed, user.id=8441761690594b9d8a423147d5d6d62d) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:39.477635 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19b6c5-9672-438f-af6a-dad40a641f75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:39.479524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19b6c5-9672-438f-af6a-dad40a641f75 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:39.495026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 83/177] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:31:39 2022] GET /identity/v3/role_assignments?user.id=8441761690594b9d8a423147d5d6d62d&scope.project.id=e263f641bede4564a5855da9ec8514ed&role.id=96c584428d924fdba9671949f760c406 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:39.600984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09029916-ecc0-4185-bd78-d1521cd10859 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:39.650534 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee091f7-9811-4720-b8eb-a7873c5283f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:39.651207 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee091f7-9811-4720-b8eb-a7873c5283f4 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:39.651515 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee091f7-9811-4720-b8eb-a7873c5283f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:39.651794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee091f7-9811-4720-b8eb-a7873c5283f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:39.747263 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee091f7-9811-4720-b8eb-a7873c5283f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:39.748697 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee091f7-9811-4720-b8eb-a7873c5283f4 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:39.784813 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:39.866499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 95/178] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:31:39.872155 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19b6c5-9672-438f-af6a-dad40a641f75 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:39.919692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582143de-5995-47bd-b3f7-877b72f765cc None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:39.920001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582143de-5995-47bd-b3f7-877b72f765cc None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:39.920283 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582143de-5995-47bd-b3f7-877b72f765cc None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:39.921923 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 96/179] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:39.927126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71830440-115a-4bfa-af9d-f9b9204c02b1 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:39.927493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71830440-115a-4bfa-af9d-f9b9204c02b1 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:39.927814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71830440-115a-4bfa-af9d-f9b9204c02b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:39.957578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-71830440-115a-4bfa-af9d-f9b9204c02b1 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:39.959778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-71830440-115a-4bfa-af9d-f9b9204c02b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:40.011208 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:40.054225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 97/180] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 22 11:31:40.079230 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19b6c5-9672-438f-af6a-dad40a641f75 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:40.094863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71830440-115a-4bfa-af9d-f9b9204c02b1 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:40.116577 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3035b-272f-4908-96bd-2675c4082870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:40.117444 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3035b-272f-4908-96bd-2675c4082870 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:40.117723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3035b-272f-4908-96bd-2675c4082870 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:40.117994 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3035b-272f-4908-96bd-2675c4082870 None admin] PATH_INFO: `/v3/projects/0ce6633b03a947ecb728e95eea7a60b5` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:40.119298 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3035b-272f-4908-96bd-2675c4082870 None admin] RBAC: Authorizing `identity:get_project(project_id=0ce6633b03a947ecb728e95eea7a60b5)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:40.120837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3035b-272f-4908-96bd-2675c4082870 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:40.122720 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 84/181] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:39 2022] GET /identity/v3/projects/0ce6633b03a947ecb728e95eea7a60b5 => generated 312 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:40.128289 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3035b-272f-4908-96bd-2675c4082870 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:40.236281 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3035b-272f-4908-96bd-2675c4082870 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:40.258084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:40.269799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:40.277251 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd4e07-7fe8-48c5-b37c-543fa11096d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:40.278331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd4e07-7fe8-48c5-b37c-543fa11096d4 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:40.278808 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd4e07-7fe8-48c5-b37c-543fa11096d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:40.279294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd4e07-7fe8-48c5-b37c-543fa11096d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:40.280384 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd4e07-7fe8-48c5-b37c-543fa11096d4 None admin] RBAC: Adding query filter params (role.id=f2336919c5104f6a8210e99994f4d455, scope.project.id=0ce6633b03a947ecb728e95eea7a60b5, user.id=d6d9a2f332cf40a6b5bc0d534f1eab4b) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:40.280914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd4e07-7fe8-48c5-b37c-543fa11096d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:40.282573 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd4e07-7fe8-48c5-b37c-543fa11096d4 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:40.290095 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 85/182] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:31:40 2022] GET /identity/v3/role_assignments?user.id=d6d9a2f332cf40a6b5bc0d534f1eab4b&scope.project.id=0ce6633b03a947ecb728e95eea7a60b5&role.id=f2336919c5104f6a8210e99994f4d455 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:40.348753 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71830440-115a-4bfa-af9d-f9b9204c02b1 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:40.421294 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1441d0-d310-42e9-9d77-51cb9818ab4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:40.422123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1441d0-d310-42e9-9d77-51cb9818ab4a None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:40.422407 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1441d0-d310-42e9-9d77-51cb9818ab4a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:40.422680 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1441d0-d310-42e9-9d77-51cb9818ab4a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:40.444645 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1441d0-d310-42e9-9d77-51cb9818ab4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:40.452533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-2f1441d0-d310-42e9-9d77-51cb9818ab4a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 22 11:31:40.452533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:40.452533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:40.452533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:40.452533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:40.452533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:40.452533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:40.452533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:40.452533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:40.452533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:40.452533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:40.452533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:40.452533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:40.452533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:40.452533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:40.452533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:40.453474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:40.453474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:40.453474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:40.453474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:40.453474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:40.453474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:40.453474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:40.453474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:40.453474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:40.453474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:40.453474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:40.453474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:40.453474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:40.453474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:40.453474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:40.453474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:40.454260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:40.454260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:40.454260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:40.454260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:40.454260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:40.454260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:40.454260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:40.454260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:40.454260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:40.454260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:40.454260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 22 11:31:40.454260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:31:40.454260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 98/183] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:31:40.459807 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd4e07-7fe8-48c5-b37c-543fa11096d4 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:40.573832 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd4e07-7fe8-48c5-b37c-543fa11096d4 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:40.610259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac28ef-d214-40a9-b51f-47cfb226d942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:40.611103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5ca0a-1fda-4993-9106-e8a9f50c4f17 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:40.611550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5ca0a-1fda-4993-9106-e8a9f50c4f17 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:40.611707 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5ca0a-1fda-4993-9106-e8a9f50c4f17 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:40.612177 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac28ef-d214-40a9-b51f-47cfb226d942 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:40.612650 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac28ef-d214-40a9-b51f-47cfb226d942 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:40.613102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 99/184] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:31:40.613298 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac28ef-d214-40a9-b51f-47cfb226d942 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:40.614842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac28ef-d214-40a9-b51f-47cfb226d942 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:40.615360 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac28ef-d214-40a9-b51f-47cfb226d942 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:40.616970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac28ef-d214-40a9-b51f-47cfb226d942 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:40.619978 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71970e-64bc-4c20-becc-f6b6c8eea449 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:40.620217 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71970e-64bc-4c20-becc-f6b6c8eea449 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:40.620481 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71970e-64bc-4c20-becc-f6b6c8eea449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:40.627485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 86/185] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:40.636372 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac28ef-d214-40a9-b51f-47cfb226d942 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:40.646297 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-da71970e-64bc-4c20-becc-f6b6c8eea449 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:40.647547 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-da71970e-64bc-4c20-becc-f6b6c8eea449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:40.719315 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:40.742506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 100/186] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:40.769199 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71970e-64bc-4c20-becc-f6b6c8eea449 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:40.774416 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac28ef-d214-40a9-b51f-47cfb226d942 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:40.818640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc56eee-ae8d-4b7b-9417-5db601279680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:40.820697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc56eee-ae8d-4b7b-9417-5db601279680 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:40.821000 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc56eee-ae8d-4b7b-9417-5db601279680 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:40.821288 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc56eee-ae8d-4b7b-9417-5db601279680 None admin] PATH_INFO: `/v3/users/b3f43b29442d486daec5c2b648aa8069` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:40.822567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc56eee-ae8d-4b7b-9417-5db601279680 None admin] RBAC: Authorizing `identity:get_user(user_id=b3f43b29442d486daec5c2b648aa8069)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:40.824213 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc56eee-ae8d-4b7b-9417-5db601279680 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:40.826618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 87/187] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:40 2022] GET /identity/v3/users/b3f43b29442d486daec5c2b648aa8069 => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:40.828400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f2da9-c354-4f98-8b9a-8eeec1752737 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:40.828680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f2da9-c354-4f98-8b9a-8eeec1752737 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:40.828966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f2da9-c354-4f98-8b9a-8eeec1752737 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:40.830873 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 88/188] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:40.833298 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f2da9-c354-4f98-8b9a-8eeec1752737 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:40.836554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:40.893171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71970e-64bc-4c20-becc-f6b6c8eea449 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:40.934566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc419e-5de9-4a47-be4b-dc01442dd0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:40.935461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc419e-5de9-4a47-be4b-dc01442dd0f3 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:40.935748 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc419e-5de9-4a47-be4b-dc01442dd0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:40.936026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc419e-5de9-4a47-be4b-dc01442dd0f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:40.945318 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f2da9-c354-4f98-8b9a-8eeec1752737 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:40.947719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc419e-5de9-4a47-be4b-dc01442dd0f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:40.955902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-06dc419e-5de9-4a47-be4b-dc01442dd0f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:31:40.955902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:40.955902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:40.955902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:40.955902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:40.955902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:40.955902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:40.955902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:40.955902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:40.955902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:40.955902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:40.955902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:40.955902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:40.955902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:40.955902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:40.955902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:40.956836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:40.956836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:40.956836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:40.956836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:40.956836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:40.956836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:40.956836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:40.956836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:40.956836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:40.956836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:40.956836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:40.956836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:40.956836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:40.956836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:40.956836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:40.956836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:40.963507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:40.963507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:40.963507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:40.963507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:40.963507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:40.963507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:40.963507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:40.963507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:40.963507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:40.963507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:40.963507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:31:40.963507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:31:40.963507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 101/189] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:31:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:40.963507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd98ac6-061b-4d53-90ad-8675fa5f1f17 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:40.964429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd98ac6-061b-4d53-90ad-8675fa5f1f17 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:40.964429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd98ac6-061b-4d53-90ad-8675fa5f1f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:40.970523 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:40.978728 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-ffd98ac6-061b-4d53-90ad-8675fa5f1f17 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:40.979964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ffd98ac6-061b-4d53-90ad-8675fa5f1f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:40.982106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663d0d4-d730-4819-940e-37ec44057bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:40.982757 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663d0d4-d730-4819-940e-37ec44057bcf None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:40.983043 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663d0d4-d730-4819-940e-37ec44057bcf None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:40.983321 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663d0d4-d730-4819-940e-37ec44057bcf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:41.047842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 102/190] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:41.049468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4be69-b988-4146-aec8-dd29aad8b974 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:41.049758 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4be69-b988-4146-aec8-dd29aad8b974 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:41.050031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4be69-b988-4146-aec8-dd29aad8b974 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:41.051311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 103/191] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:41.052050 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4be69-b988-4146-aec8-dd29aad8b974 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:41.052393 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663d0d4-d730-4819-940e-37ec44057bcf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:41.053894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663d0d4-d730-4819-940e-37ec44057bcf None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:41.101748 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:41.154471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 89/192] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 22 11:31:41.155562 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af8181-8cc0-457c-887b-10f513fee108 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:41.156104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af8181-8cc0-457c-887b-10f513fee108 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:41.156104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af8181-8cc0-457c-887b-10f513fee108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:41.168526 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4be69-b988-4146-aec8-dd29aad8b974 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:41.174761 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d6af8181-8cc0-457c-887b-10f513fee108 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:41.175983 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d6af8181-8cc0-457c-887b-10f513fee108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:41.193236 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:41.202122 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21350b5b-1dcb-4fa5-958f-169e1d6fd19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:41.202792 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21350b5b-1dcb-4fa5-958f-169e1d6fd19a None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:41.203314 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21350b5b-1dcb-4fa5-958f-169e1d6fd19a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:41.203314 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21350b5b-1dcb-4fa5-958f-169e1d6fd19a None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:41.204106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21350b5b-1dcb-4fa5-958f-169e1d6fd19a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:41.204432 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21350b5b-1dcb-4fa5-958f-169e1d6fd19a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:41.205801 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21350b5b-1dcb-4fa5-958f-169e1d6fd19a None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:41.215067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 104/193] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:41.215908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21350b5b-1dcb-4fa5-958f-169e1d6fd19a None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:41.244521 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 90/194] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:41.252001 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af8181-8cc0-457c-887b-10f513fee108 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:41.330740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21350b5b-1dcb-4fa5-958f-169e1d6fd19a None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:41.332448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:41.343397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af8181-8cc0-457c-887b-10f513fee108 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:41.366650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145b605-724a-4c24-b850-854cd9ce527e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:41.367375 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145b605-724a-4c24-b850-854cd9ce527e None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:41.367669 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145b605-724a-4c24-b850-854cd9ce527e None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:41.367939 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145b605-724a-4c24-b850-854cd9ce527e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:41.369095 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3145b605-724a-4c24-b850-854cd9ce527e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:41.370535 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3145b605-724a-4c24-b850-854cd9ce527e None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:41.372182 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 105/195] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:31:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:41.372936 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145b605-724a-4c24-b850-854cd9ce527e None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:41.382098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b1279-3ab7-4a3f-a32d-0e2ecffd2270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:41.382959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b1279-3ab7-4a3f-a32d-0e2ecffd2270 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:41.383240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b1279-3ab7-4a3f-a32d-0e2ecffd2270 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:41.383546 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b1279-3ab7-4a3f-a32d-0e2ecffd2270 None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:41.384736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b1279-3ab7-4a3f-a32d-0e2ecffd2270 None admin] RBAC: Authorizing `identity:get_project(project_id=e263f641bede4564a5855da9ec8514ed)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:41.386180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b1279-3ab7-4a3f-a32d-0e2ecffd2270 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:41.387915 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 91/196] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:41 2022] GET /identity/v3/projects/e263f641bede4564a5855da9ec8514ed => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:41.387915 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b1279-3ab7-4a3f-a32d-0e2ecffd2270 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:41.414837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:41.479757 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145b605-724a-4c24-b850-854cd9ce527e None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:41.492859 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b1279-3ab7-4a3f-a32d-0e2ecffd2270 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:41.516697 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d7ea1-85f5-474e-8208-2fac0e66a143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:41.517567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d7ea1-85f5-474e-8208-2fac0e66a143 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:41.517852 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d7ea1-85f5-474e-8208-2fac0e66a143 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:41.518129 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d7ea1-85f5-474e-8208-2fac0e66a143 None admin] PATH_INFO: `/v3/users/e473cc17a9b64406b1aa1dac6b9ff5db` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:41.527254 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b072c-a078-4e6b-b806-efe541c4c137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:41.528107 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b072c-a078-4e6b-b806-efe541c4c137 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:41.528393 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b072c-a078-4e6b-b806-efe541c4c137 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:41.528658 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b072c-a078-4e6b-b806-efe541c4c137 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:41.532143 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d7ea1-85f5-474e-8208-2fac0e66a143 None admin] RBAC: Authorizing `identity:get_user(user_id=e473cc17a9b64406b1aa1dac6b9ff5db)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:41.533602 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d7ea1-85f5-474e-8208-2fac0e66a143 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:41.535313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 106/197] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:41 2022] GET /identity/v3/users/e473cc17a9b64406b1aa1dac6b9ff5db => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:41.536086 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d7ea1-85f5-474e-8208-2fac0e66a143 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:41.546893 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b072c-a078-4e6b-b806-efe541c4c137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:41.555598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-b95b072c-a078-4e6b-b806-efe541c4c137 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:31:41.555598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:41.555598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:41.555598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:41.555598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:41.555598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:41.555598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:41.555598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:41.555598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:41.555598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:41.555598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:41.555598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:41.555598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:41.555598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:41.555598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:41.555598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:41.556726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:41.556726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:41.556726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:41.556726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:41.556726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:41.556726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:41.556726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:41.556726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:41.556726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:41.556726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:41.556726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:41.556726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:41.556726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:41.556726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:41.556726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:41.556726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:41.557710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:41.557710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:41.557710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:41.557710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:41.557710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:41.557710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:41.557710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:41.557710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:41.557710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:41.557710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:41.557710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:31:41.557710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:41.557710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 92/198] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:31:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:41.557710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b072c-a078-4e6b-b806-efe541c4c137 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:41.568059 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:41.641249 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d7ea1-85f5-474e-8208-2fac0e66a143 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:41.642640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:41.663499 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b072c-a078-4e6b-b806-efe541c4c137 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:41.676214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b51796-251d-4ba2-8835-6eba078e6943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:41.676918 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b51796-251d-4ba2-8835-6eba078e6943 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:41.677203 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b51796-251d-4ba2-8835-6eba078e6943 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:41.677484 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b51796-251d-4ba2-8835-6eba078e6943 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:41.678192 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b51796-251d-4ba2-8835-6eba078e6943 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:41.679661 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b51796-251d-4ba2-8835-6eba078e6943 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:41.698739 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f7231-ea60-4f1a-a649-b6cb9ffb6a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:41.699579 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 107/199] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jun 22 11:31:41 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:31:41.699712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f7231-ea60-4f1a-a649-b6cb9ffb6a96 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:41.699991 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f7231-ea60-4f1a-a649-b6cb9ffb6a96 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:41.700268 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f7231-ea60-4f1a-a649-b6cb9ffb6a96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:41.700411 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b51796-251d-4ba2-8835-6eba078e6943 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:41.701167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f7231-ea60-4f1a-a649-b6cb9ffb6a96 None admin] RBAC: Adding query filter params (role.id=2b63bb896abc4a22ba92b589f8ba2a73, scope.project.id=e263f641bede4564a5855da9ec8514ed, user.id=b3f43b29442d486daec5c2b648aa8069) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:41.701481 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f7231-ea60-4f1a-a649-b6cb9ffb6a96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:41.702966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f7231-ea60-4f1a-a649-b6cb9ffb6a96 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:41.710320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 93/200] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:31:41 2022] GET /identity/v3/role_assignments?user.id=b3f43b29442d486daec5c2b648aa8069&scope.project.id=e263f641bede4564a5855da9ec8514ed&role.id=2b63bb896abc4a22ba92b589f8ba2a73 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:41.711546 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f7231-ea60-4f1a-a649-b6cb9ffb6a96 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:41.846355 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:41.855843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f7231-ea60-4f1a-a649-b6cb9ffb6a96 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:41.872198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b51796-251d-4ba2-8835-6eba078e6943 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:41.905547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d67d0-45ef-4203-9799-13c75aff72a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:41.906631 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d67d0-45ef-4203-9799-13c75aff72a2 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:41.907052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d67d0-45ef-4203-9799-13c75aff72a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:41.907341 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d67d0-45ef-4203-9799-13c75aff72a2 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:41.908423 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d67d0-45ef-4203-9799-13c75aff72a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:41.909179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d67d0-45ef-4203-9799-13c75aff72a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:41.910613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d67d0-45ef-4203-9799-13c75aff72a2 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:41.915141 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0c362-9aee-4c1c-9c32-57c45912415f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:41.915973 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0c362-9aee-4c1c-9c32-57c45912415f None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:41.916219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0c362-9aee-4c1c-9c32-57c45912415f None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:41.917033 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c0c362-9aee-4c1c-9c32-57c45912415f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:41.920147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 94/201] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:31:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:41.925855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d67d0-45ef-4203-9799-13c75aff72a2 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:41.944342 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:42.010062 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0c362-9aee-4c1c-9c32-57c45912415f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:42.011673 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c0c362-9aee-4c1c-9c32-57c45912415f None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:42.070906 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d67d0-45ef-4203-9799-13c75aff72a2 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:42.117476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d76a40-53e3-4ccb-9ec6-de92ec52d705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:42.118632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d76a40-53e3-4ccb-9ec6-de92ec52d705 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:42.118920 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d76a40-53e3-4ccb-9ec6-de92ec52d705 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:42.119199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d76a40-53e3-4ccb-9ec6-de92ec52d705 None admin] PATH_INFO: `/v3/users/e473cc17a9b64406b1aa1dac6b9ff5db` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:42.119929 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:42.121974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d76a40-53e3-4ccb-9ec6-de92ec52d705 None admin] RBAC: Authorizing `identity:get_user(user_id=e473cc17a9b64406b1aa1dac6b9ff5db)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:42.123730 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d76a40-53e3-4ccb-9ec6-de92ec52d705 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:42.126901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 95/202] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:41 2022] GET /identity/v3/users/e473cc17a9b64406b1aa1dac6b9ff5db => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:42.132964 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d76a40-53e3-4ccb-9ec6-de92ec52d705 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:42.140956 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 108/203] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:31:42.148448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0c362-9aee-4c1c-9c32-57c45912415f None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:42.213198 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:42.238149 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d76a40-53e3-4ccb-9ec6-de92ec52d705 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:42.261805 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c0c362-9aee-4c1c-9c32-57c45912415f None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:42.278415 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890019b9-77fa-4544-a8df-e076de6a62eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:42.279195 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890019b9-77fa-4544-a8df-e076de6a62eb None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:42.279585 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890019b9-77fa-4544-a8df-e076de6a62eb None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:42.279943 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890019b9-77fa-4544-a8df-e076de6a62eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:42.300328 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451dcd8-f66e-4e18-bcba-6de7d274771e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:42.301379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451dcd8-f66e-4e18-bcba-6de7d274771e None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:42.301750 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451dcd8-f66e-4e18-bcba-6de7d274771e None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:42.302619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451dcd8-f66e-4e18-bcba-6de7d274771e None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:42.303940 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451dcd8-f66e-4e18-bcba-6de7d274771e None admin] RBAC: Authorizing `identity:get_project(project_id=a89544d6d9d14e759c077d7175365dca)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:42.305509 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451dcd8-f66e-4e18-bcba-6de7d274771e None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:42.312488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 109/204] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:42 2022] GET /identity/v3/projects/a89544d6d9d14e759c077d7175365dca => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:42.313891 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451dcd8-f66e-4e18-bcba-6de7d274771e None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:42.358139 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890019b9-77fa-4544-a8df-e076de6a62eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:42.359782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890019b9-77fa-4544-a8df-e076de6a62eb None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:42.372631 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:42.427481 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451dcd8-f66e-4e18-bcba-6de7d274771e None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:42.443383 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:42.475656 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 96/205] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:31:42.479081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890019b9-77fa-4544-a8df-e076de6a62eb None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:42.496119 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42844cd-ea44-480c-a546-2a6b934f1783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:42.497879 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42844cd-ea44-480c-a546-2a6b934f1783 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:42.498388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42844cd-ea44-480c-a546-2a6b934f1783 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:42.499133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42844cd-ea44-480c-a546-2a6b934f1783 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:42.500627 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42844cd-ea44-480c-a546-2a6b934f1783 None admin] RBAC: Adding query filter params (role.id=96c584428d924fdba9671949f760c406, scope.project.id=a89544d6d9d14e759c077d7175365dca, user.id=e473cc17a9b64406b1aa1dac6b9ff5db) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:42.501151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42844cd-ea44-480c-a546-2a6b934f1783 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:42.503327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42844cd-ea44-480c-a546-2a6b934f1783 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:42.512436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 110/206] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:31:42 2022] GET /identity/v3/role_assignments?user.id=e473cc17a9b64406b1aa1dac6b9ff5db&scope.project.id=a89544d6d9d14e759c077d7175365dca&role.id=96c584428d924fdba9671949f760c406 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:31:42.595191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890019b9-77fa-4544-a8df-e076de6a62eb None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:42.635156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ff2e3-c82d-43e2-98a4-82c39f5270b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:42.636335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ff2e3-c82d-43e2-98a4-82c39f5270b2 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:42.636885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ff2e3-c82d-43e2-98a4-82c39f5270b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:42.637311 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ff2e3-c82d-43e2-98a4-82c39f5270b2 None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:42.638838 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ff2e3-c82d-43e2-98a4-82c39f5270b2 None admin] RBAC: Authorizing `identity:get_project(project_id=a89544d6d9d14e759c077d7175365dca)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:42.640802 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ff2e3-c82d-43e2-98a4-82c39f5270b2 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:42.643052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 97/207] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:42 2022] GET /identity/v3/projects/a89544d6d9d14e759c077d7175365dca => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:42.649795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42844cd-ea44-480c-a546-2a6b934f1783 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:42.758254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42844cd-ea44-480c-a546-2a6b934f1783 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:42.776324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:42.795671 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78808420-6fba-488d-93f9-f8a2f53e5544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:42.796553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78808420-6fba-488d-93f9-f8a2f53e5544 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:42.796984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78808420-6fba-488d-93f9-f8a2f53e5544 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:42.797405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78808420-6fba-488d-93f9-f8a2f53e5544 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:42.798408 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78808420-6fba-488d-93f9-f8a2f53e5544 None admin] RBAC: Adding query filter params (role.id=b4be7021e4104740b8b0b3b40ecd3d59, scope.project.id=a89544d6d9d14e759c077d7175365dca, user.id=e473cc17a9b64406b1aa1dac6b9ff5db) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:42.798863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78808420-6fba-488d-93f9-f8a2f53e5544 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:42.800469 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78808420-6fba-488d-93f9-f8a2f53e5544 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:42.806403 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 111/208] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:31:42 2022] GET /identity/v3/role_assignments?user.id=e473cc17a9b64406b1aa1dac6b9ff5db&scope.project.id=a89544d6d9d14e759c077d7175365dca&role.id=b4be7021e4104740b8b0b3b40ecd3d59 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:44.011199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dbbf81-68e6-4321-964b-7a6d775d261f None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:44.011720 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dbbf81-68e6-4321-964b-7a6d775d261f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:44.011796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dbbf81-68e6-4321-964b-7a6d775d261f None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:44.013221 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 98/209] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:44.018744 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e28b36-69cd-44d6-963e-92437c7ae7f9 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:44.019613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e28b36-69cd-44d6-963e-92437c7ae7f9 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:44.020098 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e28b36-69cd-44d6-963e-92437c7ae7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:44.038384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-d8e28b36-69cd-44d6-963e-92437c7ae7f9 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:44.039632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d8e28b36-69cd-44d6-963e-92437c7ae7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:44.046909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e81f6-f6b9-4312-9608-9e880a8cd92c None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:44.047200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e81f6-f6b9-4312-9608-9e880a8cd92c None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:44.047507 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e81f6-f6b9-4312-9608-9e880a8cd92c None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:44.049131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 99/210] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:44.054872 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04960fa-ac7e-4a10-b656-9b483bf5a23a None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:44.055166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04960fa-ac7e-4a10-b656-9b483bf5a23a None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:44.055481 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04960fa-ac7e-4a10-b656-9b483bf5a23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:44.072551 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-b04960fa-ac7e-4a10-b656-9b483bf5a23a None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:44.073749 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b04960fa-ac7e-4a10-b656-9b483bf5a23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:44.115460 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:44.120433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 112/211] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:44.145487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e28b36-69cd-44d6-963e-92437c7ae7f9 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:44.150359 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 100/212] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 22 11:31:44.173640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04960fa-ac7e-4a10-b656-9b483bf5a23a None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:44.266922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e28b36-69cd-44d6-963e-92437c7ae7f9 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:44.278577 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:44.292364 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04960fa-ac7e-4a10-b656-9b483bf5a23a None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:44.312402 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7375bf-3aa4-4570-806e-09c6abb178ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:44.313266 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7375bf-3aa4-4570-806e-09c6abb178ef None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:44.313552 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7375bf-3aa4-4570-806e-09c6abb178ef None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:44.313856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7375bf-3aa4-4570-806e-09c6abb178ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:44.315114 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7375bf-3aa4-4570-806e-09c6abb178ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:44.316670 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7375bf-3aa4-4570-806e-09c6abb178ef None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:44.319392 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 113/213] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:31:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:31:44.327884 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7375bf-3aa4-4570-806e-09c6abb178ef None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:44.345163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a6f20-3fca-4afd-812b-96ebc1903ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:44.346003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a6f20-3fca-4afd-812b-96ebc1903ff5 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:44.346285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a6f20-3fca-4afd-812b-96ebc1903ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:44.346561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a6f20-3fca-4afd-812b-96ebc1903ff5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:44.361463 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a6f20-3fca-4afd-812b-96ebc1903ff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:44.372641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-5a6a6f20-3fca-4afd-812b-96ebc1903ff5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:44.372641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:44.372641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:44.372641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:44.372641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:44.372641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:44.372641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:44.372641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:44.372641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:44.372641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:44.372641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:44.372641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:44.372641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:44.372641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:44.372641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:44.372641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:44.373578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:44.373578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:44.373578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:44.373578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:44.373578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:44.373578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:44.373578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:44.373578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:44.373578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:44.373578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:44.373578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:44.373578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:44.373578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:44.373578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:44.373578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:44.373578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:44.374362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:44.374362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:44.374362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:44.374362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:44.374362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:44.374362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:44.374362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:44.374362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:44.374362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:44.374362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:44.374362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:44.374362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:44.374362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 101/214] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:44.382009 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a6f20-3fca-4afd-812b-96ebc1903ff5 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:44.417798 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:44.454473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7375bf-3aa4-4570-806e-09c6abb178ef None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:44.489324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a6f20-3fca-4afd-812b-96ebc1903ff5 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:44.492312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eac723-5656-4358-852c-471f229ee3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:44.493256 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eac723-5656-4358-852c-471f229ee3f1 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:44.493740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eac723-5656-4358-852c-471f229ee3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:44.494232 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eac723-5656-4358-852c-471f229ee3f1 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:44.495156 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eac723-5656-4358-852c-471f229ee3f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:44.496995 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eac723-5656-4358-852c-471f229ee3f1 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:44.521979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 114/215] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jun 22 11:31:44 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:31:44.540251 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27222234-94d5-452d-9eb1-db6f76118aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:44.541335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27222234-94d5-452d-9eb1-db6f76118aae None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:44.541831 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27222234-94d5-452d-9eb1-db6f76118aae None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:44.542359 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27222234-94d5-452d-9eb1-db6f76118aae None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:44.543517 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27222234-94d5-452d-9eb1-db6f76118aae None admin] RBAC: Adding query filter params (name=member) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:44.544104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27222234-94d5-452d-9eb1-db6f76118aae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:44.545842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27222234-94d5-452d-9eb1-db6f76118aae None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:44.556942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 102/216] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:44.562736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eac723-5656-4358-852c-471f229ee3f1 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:44.599541 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d961e-007c-4da1-80fc-dc422289cfa2 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:44.600163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d961e-007c-4da1-80fc-dc422289cfa2 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:44.600645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d961e-007c-4da1-80fc-dc422289cfa2 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:44.602371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 103/217] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:31:44.608302 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06a973-d1a2-4443-ac52-85f9e6ca8afb None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:44.608595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06a973-d1a2-4443-ac52-85f9e6ca8afb None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:44.608846 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06a973-d1a2-4443-ac52-85f9e6ca8afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:44.609197 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:44.632881 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-fe06a973-d1a2-4443-ac52-85f9e6ca8afb None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:44.634473 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-fe06a973-d1a2-4443-ac52-85f9e6ca8afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:44.692021 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eac723-5656-4358-852c-471f229ee3f1 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:44.741132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 104/218] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 22 11:31:44.743690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55380a5a-5b9a-4b56-94e1-d82e1af64928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:44.744938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55380a5a-5b9a-4b56-94e1-d82e1af64928 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:44.745434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55380a5a-5b9a-4b56-94e1-d82e1af64928 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:44.745947 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55380a5a-5b9a-4b56-94e1-d82e1af64928 None admin] PATH_INFO: `/v3/users/b0232b4d01e84bb1a0a401bec86e975c` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:44.762386 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55380a5a-5b9a-4b56-94e1-d82e1af64928 None admin] RBAC: Authorizing `identity:get_user(user_id=b0232b4d01e84bb1a0a401bec86e975c)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:44.764531 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06a973-d1a2-4443-ac52-85f9e6ca8afb None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:44.767959 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55380a5a-5b9a-4b56-94e1-d82e1af64928 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:44.769923 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 115/219] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:44 2022] GET /identity/v3/users/b0232b4d01e84bb1a0a401bec86e975c => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:44.777470 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55380a5a-5b9a-4b56-94e1-d82e1af64928 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:44.801196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:44.910012 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:44.914552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06a973-d1a2-4443-ac52-85f9e6ca8afb None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:44.921958 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55380a5a-5b9a-4b56-94e1-d82e1af64928 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:44.960250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399cd6c-2fe7-422c-a3b7-784e9c3b1623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:44.960878 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399cd6c-2fe7-422c-a3b7-784e9c3b1623 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:44.961005 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399cd6c-2fe7-422c-a3b7-784e9c3b1623 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:44.961263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399cd6c-2fe7-422c-a3b7-784e9c3b1623 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:44.971769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7599421-c31e-4b58-a399-bf7f84654137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:44.972429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7599421-c31e-4b58-a399-bf7f84654137 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:44.972714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7599421-c31e-4b58-a399-bf7f84654137 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:44.972987 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7599421-c31e-4b58-a399-bf7f84654137 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:44.974282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399cd6c-2fe7-422c-a3b7-784e9c3b1623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:44.982028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-d399cd6c-2fe7-422c-a3b7-784e9c3b1623 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:31:44.982028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:44.982028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:44.982028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:44.982028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:44.982028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:44.982028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:44.982028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:44.982028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:44.982028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:44.982028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:44.982028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:44.982028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:44.982028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:44.982028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:44.982028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:44.982951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:44.982951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:44.982951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:44.982951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:44.982951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:44.982951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:44.982951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:44.982951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:44.982951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:44.982951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:44.982951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:44.982951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:44.982951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:44.982951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:44.982951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:44.982951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:44.984244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:44.984244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:44.984244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:44.984244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:44.984244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:44.984244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:44.984244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:44.984244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:44.984244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:44.984244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:44.984244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:31:44.984244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:44.984244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 105/220] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:31:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:31:44.984972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94f71e-5876-44e3-b750-ee6494b3b6dd None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:44.984972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94f71e-5876-44e3-b750-ee6494b3b6dd None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:44.985088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94f71e-5876-44e3-b750-ee6494b3b6dd None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:44.990647 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 106/221] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:44.992699 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94f71e-5876-44e3-b750-ee6494b3b6dd None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:45.068795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7599421-c31e-4b58-a399-bf7f84654137 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:45.071774 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7599421-c31e-4b58-a399-bf7f84654137 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:45.091822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:45.110996 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94f71e-5876-44e3-b750-ee6494b3b6dd None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:45.147628 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b2566-737f-4693-bc84-7e4b0faa3105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:45.148549 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b2566-737f-4693-bc84-7e4b0faa3105 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:45.148981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b2566-737f-4693-bc84-7e4b0faa3105 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:45.149402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b2566-737f-4693-bc84-7e4b0faa3105 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:45.150309 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b2566-737f-4693-bc84-7e4b0faa3105 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:45.150827 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b2566-737f-4693-bc84-7e4b0faa3105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:45.152447 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b2566-737f-4693-bc84-7e4b0faa3105 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:45.162034 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 107/222] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:45.163650 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e91dd-32b9-4135-99d2-be6e370de427 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:45.164107 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e91dd-32b9-4135-99d2-be6e370de427 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:45.164573 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e91dd-32b9-4135-99d2-be6e370de427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:45.186434 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f30e91dd-32b9-4135-99d2-be6e370de427 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:45.188012 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f30e91dd-32b9-4135-99d2-be6e370de427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:45.199929 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:45.217183 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 116/223] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:31:45.218581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7599421-c31e-4b58-a399-bf7f84654137 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:45.267915 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 108/224] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:45.276133 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e91dd-32b9-4135-99d2-be6e370de427 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:45.353038 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7599421-c31e-4b58-a399-bf7f84654137 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:45.389327 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:45.404600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba5faa-f130-4dbb-b680-7d5ac1024613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:45.406426 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba5faa-f130-4dbb-b680-7d5ac1024613 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:45.407122 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba5faa-f130-4dbb-b680-7d5ac1024613 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:45.407578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba5faa-f130-4dbb-b680-7d5ac1024613 None admin] PATH_INFO: `/v3/users/e473cc17a9b64406b1aa1dac6b9ff5db` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:45.409973 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba5faa-f130-4dbb-b680-7d5ac1024613 None admin] RBAC: Authorizing `identity:get_user(user_id=e473cc17a9b64406b1aa1dac6b9ff5db)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:45.412342 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba5faa-f130-4dbb-b680-7d5ac1024613 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:45.414511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 117/225] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:45 2022] GET /identity/v3/users/e473cc17a9b64406b1aa1dac6b9ff5db => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:45.415456 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba5faa-f130-4dbb-b680-7d5ac1024613 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:45.419191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e91dd-32b9-4135-99d2-be6e370de427 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:45.465543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7feeaa5-2d66-4a35-add5-c2ce39d384b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:45.466414 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7feeaa5-2d66-4a35-add5-c2ce39d384b7 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:45.466696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7feeaa5-2d66-4a35-add5-c2ce39d384b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:45.466969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7feeaa5-2d66-4a35-add5-c2ce39d384b7 None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:45.468641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7feeaa5-2d66-4a35-add5-c2ce39d384b7 None admin] RBAC: Authorizing `identity:get_project(project_id=a89544d6d9d14e759c077d7175365dca)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:45.470242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7feeaa5-2d66-4a35-add5-c2ce39d384b7 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:45.472953 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 109/226] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:45 2022] GET /identity/v3/projects/a89544d6d9d14e759c077d7175365dca => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:45.476141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7feeaa5-2d66-4a35-add5-c2ce39d384b7 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:45.501863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:45.551556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba5faa-f130-4dbb-b680-7d5ac1024613 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:45.584549 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7feeaa5-2d66-4a35-add5-c2ce39d384b7 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:45.589420 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ca989-f754-4053-a04f-111171f2a760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:45.591004 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ca989-f754-4053-a04f-111171f2a760 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:45.591004 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ca989-f754-4053-a04f-111171f2a760 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:45.591004 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ca989-f754-4053-a04f-111171f2a760 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:45.602949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ca989-f754-4053-a04f-111171f2a760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:45.612130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-b41ca989-f754-4053-a04f-111171f2a760 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:31:45.612130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:45.612130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:45.612130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:45.612130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:45.612130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:45.612130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:45.612130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:45.612130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:45.612130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:45.612130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:45.612130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:45.612130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:45.612130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:45.612130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:45.612130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:45.613194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:45.613194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:45.613194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:45.613194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:45.613194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:45.613194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:45.613194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:45.613194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:45.613194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:45.613194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:45.613194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:45.613194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:45.613194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:45.613194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:45.613194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:45.613194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:45.619605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:45.619605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:45.619605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:45.619605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:45.619605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:45.619605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:45.619605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:45.619605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:45.619605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:45.619605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:45.619605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:31:45.619605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:31:45.619605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 118/227] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:31:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:45.619605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ca989-f754-4053-a04f-111171f2a760 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:45.632073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eea786-9714-4495-bbbf-d2cbfc56a8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:45.633202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eea786-9714-4495-bbbf-d2cbfc56a8d8 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:45.633654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eea786-9714-4495-bbbf-d2cbfc56a8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:45.634072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eea786-9714-4495-bbbf-d2cbfc56a8d8 None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:45.635479 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eea786-9714-4495-bbbf-d2cbfc56a8d8 None admin] RBAC: Authorizing `identity:get_project(project_id=a89544d6d9d14e759c077d7175365dca)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:45.637123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eea786-9714-4495-bbbf-d2cbfc56a8d8 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:45.639364 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 110/228] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:45 2022] GET /identity/v3/projects/a89544d6d9d14e759c077d7175365dca => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:45.640579 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eea786-9714-4495-bbbf-d2cbfc56a8d8 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:45.657083 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:45.742736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ca989-f754-4053-a04f-111171f2a760 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:45.748576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eea786-9714-4495-bbbf-d2cbfc56a8d8 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:45.766864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:45.784311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc11619-31f1-45f9-89a7-50e7b9be73d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:45.785229 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc11619-31f1-45f9-89a7-50e7b9be73d3 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:45.785700 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc11619-31f1-45f9-89a7-50e7b9be73d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:45.785867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9bea4-e737-4d39-8988-5548f8a97e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:45.786329 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc11619-31f1-45f9-89a7-50e7b9be73d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:45.786578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9bea4-e737-4d39-8988-5548f8a97e80 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:45.786863 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9bea4-e737-4d39-8988-5548f8a97e80 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:45.787139 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9bea4-e737-4d39-8988-5548f8a97e80 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:45.787815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc11619-31f1-45f9-89a7-50e7b9be73d3 None admin] RBAC: Adding query filter params (role.id=f2336919c5104f6a8210e99994f4d455, scope.project.id=a89544d6d9d14e759c077d7175365dca, user.id=b0232b4d01e84bb1a0a401bec86e975c) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:45.787929 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e9bea4-e737-4d39-8988-5548f8a97e80 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:45.788227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e9bea4-e737-4d39-8988-5548f8a97e80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:45.788515 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc11619-31f1-45f9-89a7-50e7b9be73d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:45.789623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e9bea4-e737-4d39-8988-5548f8a97e80 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:45.790396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc11619-31f1-45f9-89a7-50e7b9be73d3 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:45.802506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 111/229] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:31:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:45.802506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9bea4-e737-4d39-8988-5548f8a97e80 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:45.803509 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 119/230] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:31:45 2022] GET /identity/v3/role_assignments?user.id=b0232b4d01e84bb1a0a401bec86e975c&scope.project.id=a89544d6d9d14e759c077d7175365dca&role.id=f2336919c5104f6a8210e99994f4d455 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:45.809574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc11619-31f1-45f9-89a7-50e7b9be73d3 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:45.943807 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:46.003642 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9bea4-e737-4d39-8988-5548f8a97e80 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:46.043590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc11619-31f1-45f9-89a7-50e7b9be73d3 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:46.047636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dff9a5-4768-4c2d-a143-c53516d1a1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:46.048548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dff9a5-4768-4c2d-a143-c53516d1a1fb None admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:46.048828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dff9a5-4768-4c2d-a143-c53516d1a1fb None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:46.049104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dff9a5-4768-4c2d-a143-c53516d1a1fb None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca/users/e473cc17a9b64406b1aa1dac6b9ff5db/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:46.053206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dff9a5-4768-4c2d-a143-c53516d1a1fb None admin] RBAC: Authorizing `identity:create_grant(project_id=a89544d6d9d14e759c077d7175365dca, user_id=e473cc17a9b64406b1aa1dac6b9ff5db, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:46.054016 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dff9a5-4768-4c2d-a143-c53516d1a1fb None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:46.075422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 112/231] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:31:45 2022] PUT /identity/v3/projects/a89544d6d9d14e759c077d7175365dca/users/e473cc17a9b64406b1aa1dac6b9ff5db/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:31:46.106882 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8810fb0-3f2e-4c99-a7fe-6b3f48291217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:46.108568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8810fb0-3f2e-4c99-a7fe-6b3f48291217 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:46.109044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8810fb0-3f2e-4c99-a7fe-6b3f48291217 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:46.109511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8810fb0-3f2e-4c99-a7fe-6b3f48291217 None admin] PATH_INFO: `/v3/users/e473cc17a9b64406b1aa1dac6b9ff5db` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:46.112146 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8810fb0-3f2e-4c99-a7fe-6b3f48291217 None admin] RBAC: Authorizing `identity:get_user(user_id=e473cc17a9b64406b1aa1dac6b9ff5db)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:46.113981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8810fb0-3f2e-4c99-a7fe-6b3f48291217 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:46.116696 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 120/232] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:45 2022] GET /identity/v3/users/e473cc17a9b64406b1aa1dac6b9ff5db => generated 283 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:46.122703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dff9a5-4768-4c2d-a143-c53516d1a1fb None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:46.278584 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dff9a5-4768-4c2d-a143-c53516d1a1fb None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:46.317385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fda4d5-a600-4b1a-816e-d38d70bb7854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:46.318765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fda4d5-a600-4b1a-816e-d38d70bb7854 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:46.319267 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fda4d5-a600-4b1a-816e-d38d70bb7854 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:46.319815 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fda4d5-a600-4b1a-816e-d38d70bb7854 None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:46.321386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fda4d5-a600-4b1a-816e-d38d70bb7854 None admin] RBAC: Authorizing `identity:get_project(project_id=a89544d6d9d14e759c077d7175365dca)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:46.323081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fda4d5-a600-4b1a-816e-d38d70bb7854 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:46.325222 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 113/233] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:46 2022] GET /identity/v3/projects/a89544d6d9d14e759c077d7175365dca => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:46.331582 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8810fb0-3f2e-4c99-a7fe-6b3f48291217 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:46.392530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:46.446304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8810fb0-3f2e-4c99-a7fe-6b3f48291217 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:46.484140 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e62cad-6054-4a17-80d3-42f28fc18c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:46.485474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e62cad-6054-4a17-80d3-42f28fc18c8b None admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:46.486006 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e62cad-6054-4a17-80d3-42f28fc18c8b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:46.486497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e62cad-6054-4a17-80d3-42f28fc18c8b None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca/users/e473cc17a9b64406b1aa1dac6b9ff5db/roles/b4be7021e4104740b8b0b3b40ecd3d59` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:46.489072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e62cad-6054-4a17-80d3-42f28fc18c8b None admin] RBAC: Authorizing `identity:create_grant(project_id=a89544d6d9d14e759c077d7175365dca, user_id=e473cc17a9b64406b1aa1dac6b9ff5db, role_id=b4be7021e4104740b8b0b3b40ecd3d59)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:46.490841 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e62cad-6054-4a17-80d3-42f28fc18c8b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:46.512286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 121/234] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:31:46 2022] PUT /identity/v3/projects/a89544d6d9d14e759c077d7175365dca/users/e473cc17a9b64406b1aa1dac6b9ff5db/roles/b4be7021e4104740b8b0b3b40ecd3d59 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:31:47.072681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29649c9-057e-4081-ada7-326549952f41 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:47.073015 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29649c9-057e-4081-ada7-326549952f41 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:47.073242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29649c9-057e-4081-ada7-326549952f41 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:47.074933 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 114/235] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:47.080822 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc061c-204a-48cf-b815-0b16850e2cf6 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:47.081139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc061c-204a-48cf-b815-0b16850e2cf6 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:47.081384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc061c-204a-48cf-b815-0b16850e2cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:47.102260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-97dc061c-204a-48cf-b815-0b16850e2cf6 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:47.103501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-97dc061c-204a-48cf-b815-0b16850e2cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:47.178922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 122/236] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:47.204454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29649c9-057e-4081-ada7-326549952f41 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:47.208436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:47.231335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304188b-cbaf-47c7-8503-65cd0a893422 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:47.231870 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304188b-cbaf-47c7-8503-65cd0a893422 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:47.232204 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304188b-cbaf-47c7-8503-65cd0a893422 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:47.234056 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 123/237] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:47.239792 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ccc80-e7e8-4ac4-9e0d-d384dab2049e None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:47.240109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ccc80-e7e8-4ac4-9e0d-d384dab2049e None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:47.240489 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ccc80-e7e8-4ac4-9e0d-d384dab2049e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:47.274694 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-383ccc80-e7e8-4ac4-9e0d-d384dab2049e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:47.275897 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-383ccc80-e7e8-4ac4-9e0d-d384dab2049e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:47.328263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29649c9-057e-4081-ada7-326549952f41 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:47.334832 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:47.390913 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971c6fe-9504-49be-ba84-0c1cf8e759f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:47.392137 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971c6fe-9504-49be-ba84-0c1cf8e759f3 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:47.392826 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971c6fe-9504-49be-ba84-0c1cf8e759f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:47.394157 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971c6fe-9504-49be-ba84-0c1cf8e759f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:47.396438 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f971c6fe-9504-49be-ba84-0c1cf8e759f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:47.399393 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f971c6fe-9504-49be-ba84-0c1cf8e759f3 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:47.402224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 115/238] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:31:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:47.403732 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 124/239] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:47.408059 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ccc80-e7e8-4ac4-9e0d-d384dab2049e None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:47.442475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971c6fe-9504-49be-ba84-0c1cf8e759f3 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:47.591093 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:47.600339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ccc80-e7e8-4ac4-9e0d-d384dab2049e None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:47.653113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971c6fe-9504-49be-ba84-0c1cf8e759f3 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:47.668863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609df67f-d1eb-47d1-bce1-73b52befb266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:47.669552 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609df67f-d1eb-47d1-bce1-73b52befb266 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:47.669839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609df67f-d1eb-47d1-bce1-73b52befb266 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:47.670114 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609df67f-d1eb-47d1-bce1-73b52befb266 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:47.670838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609df67f-d1eb-47d1-bce1-73b52befb266 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:47.673896 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609df67f-d1eb-47d1-bce1-73b52befb266 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:47.711192 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 125/240] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jun 22 11:31:47 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:31:47.718808 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb26d87-267f-4e9c-a465-53f6c2964177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:47.719758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb26d87-267f-4e9c-a465-53f6c2964177 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:47.720050 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb26d87-267f-4e9c-a465-53f6c2964177 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:47.720328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb26d87-267f-4e9c-a465-53f6c2964177 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:47.732853 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb26d87-267f-4e9c-a465-53f6c2964177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:47.741586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-eeb26d87-267f-4e9c-a465-53f6c2964177 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 22 11:31:47.741586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:47.741586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:47.741586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:47.741586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:47.741586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:47.741586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:47.741586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:47.741586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:47.741586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:47.741586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:47.741586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:47.741586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:47.741586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:47.741586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:47.741586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:47.742543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:47.742543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:47.742543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:47.742543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:47.742543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:47.742543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:47.742543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:47.742543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:47.742543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:47.742543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:47.742543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:47.742543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:47.742543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:47.742543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:47.742543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:47.742543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:47.743331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:47.743331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:47.743331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:47.743331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:47.743331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:47.743331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:47.743331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:47.743331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:47.743331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:47.743331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:47.743331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 22 11:31:47.743331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:47.743331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 116/241] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:47.749942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609df67f-d1eb-47d1-bce1-73b52befb266 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:47.801701 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:47.910419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609df67f-d1eb-47d1-bce1-73b52befb266 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:47.962292 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150d6f0-92a5-429c-8e30-83d0ba260f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:47.963381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150d6f0-92a5-429c-8e30-83d0ba260f28 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:47.963885 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150d6f0-92a5-429c-8e30-83d0ba260f28 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:47.964348 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150d6f0-92a5-429c-8e30-83d0ba260f28 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:47.965319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150d6f0-92a5-429c-8e30-83d0ba260f28 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:47.965828 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150d6f0-92a5-429c-8e30-83d0ba260f28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:47.967504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150d6f0-92a5-429c-8e30-83d0ba260f28 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:47.977311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 126/242] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:47.982902 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb26d87-267f-4e9c-a465-53f6c2964177 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:48.130804 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb26d87-267f-4e9c-a465-53f6c2964177 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:48.168877 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:48.175934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6a76e-914f-4c35-9098-500c69c47b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:48.177777 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6a76e-914f-4c35-9098-500c69c47b19 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:48.178259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6a76e-914f-4c35-9098-500c69c47b19 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:48.178737 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead6a76e-914f-4c35-9098-500c69c47b19 None admin] PATH_INFO: `/v3/users/393f9d34e4534a2ebc37217c9e6a4ea9` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:48.203111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6a76e-914f-4c35-9098-500c69c47b19 None admin] RBAC: Authorizing `identity:get_user(user_id=393f9d34e4534a2ebc37217c9e6a4ea9)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:48.205092 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead6a76e-914f-4c35-9098-500c69c47b19 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:48.207654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 117/243] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:47 2022] GET /identity/v3/users/393f9d34e4534a2ebc37217c9e6a4ea9 => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:48.216860 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150d6f0-92a5-429c-8e30-83d0ba260f28 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:48.328180 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:48.344172 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150d6f0-92a5-429c-8e30-83d0ba260f28 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:48.401054 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d778c2-4045-44d5-90da-18bd34a10563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:48.401732 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d778c2-4045-44d5-90da-18bd34a10563 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:48.402017 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d778c2-4045-44d5-90da-18bd34a10563 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:48.402295 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d778c2-4045-44d5-90da-18bd34a10563 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:48.487912 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d778c2-4045-44d5-90da-18bd34a10563 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:48.489369 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d778c2-4045-44d5-90da-18bd34a10563 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:48.607516 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:48.623646 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 127/244] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:31:48.629348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6a76e-914f-4c35-9098-500c69c47b19 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:48.672616 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cf4ca-3efe-46a1-980d-cf7ee2e24aee None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:48.672895 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cf4ca-3efe-46a1-980d-cf7ee2e24aee None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:48.673172 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cf4ca-3efe-46a1-980d-cf7ee2e24aee None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:48.674568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 128/245] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:31:48.680496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef9b81-788b-4752-87b6-d3c39ccf2549 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:48.680789 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef9b81-788b-4752-87b6-d3c39ccf2549 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:48.681070 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef9b81-788b-4752-87b6-d3c39ccf2549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:48.701995 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-b0ef9b81-788b-4752-87b6-d3c39ccf2549 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:48.703936 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b0ef9b81-788b-4752-87b6-d3c39ccf2549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:48.800501 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead6a76e-914f-4c35-9098-500c69c47b19 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:48.818797 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 129/246] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 22 11:31:48.845427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126f6b8-29c5-4f59-9c35-288da7bb7caa None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:48.845745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126f6b8-29c5-4f59-9c35-288da7bb7caa None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:48.845992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126f6b8-29c5-4f59-9c35-288da7bb7caa None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:48.847474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 130/247] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:48.849509 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126f6b8-29c5-4f59-9c35-288da7bb7caa None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:48.888150 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136425d1-b3fe-498f-b05f-8f568099587b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:48.890332 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136425d1-b3fe-498f-b05f-8f568099587b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:48.890616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136425d1-b3fe-498f-b05f-8f568099587b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:48.890891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136425d1-b3fe-498f-b05f-8f568099587b None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:48.892286 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136425d1-b3fe-498f-b05f-8f568099587b None admin] RBAC: Authorizing `identity:get_project(project_id=a89544d6d9d14e759c077d7175365dca)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:48.893852 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136425d1-b3fe-498f-b05f-8f568099587b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:48.896632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 118/248] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:48 2022] GET /identity/v3/projects/a89544d6d9d14e759c077d7175365dca => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:48.897901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbde605-5123-411e-bc4d-b34b224ec811 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:48.898180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbde605-5123-411e-bc4d-b34b224ec811 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:48.898515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbde605-5123-411e-bc4d-b34b224ec811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:48.917356 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-5bbde605-5123-411e-bc4d-b34b224ec811 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:48.918643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5bbde605-5123-411e-bc4d-b34b224ec811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:48.984177 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:49.002212 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 119/249] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:49.003123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbde605-5123-411e-bc4d-b34b224ec811 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:49.118751 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbde605-5123-411e-bc4d-b34b224ec811 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:49.128132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126f6b8-29c5-4f59-9c35-288da7bb7caa None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:49.172809 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:49.174774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9286c-256e-4b9a-bd33-b12380929743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:49.175791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9286c-256e-4b9a-bd33-b12380929743 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:49.176443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9286c-256e-4b9a-bd33-b12380929743 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:49.178340 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9286c-256e-4b9a-bd33-b12380929743 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:49.179389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9286c-256e-4b9a-bd33-b12380929743 None admin] RBAC: Adding query filter params (role.id=2b63bb896abc4a22ba92b589f8ba2a73, scope.project.id=a89544d6d9d14e759c077d7175365dca, user.id=393f9d34e4534a2ebc37217c9e6a4ea9) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:49.179876 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9286c-256e-4b9a-bd33-b12380929743 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:49.181524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9286c-256e-4b9a-bd33-b12380929743 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:49.188928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 120/250] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:31:49 2022] GET /identity/v3/role_assignments?user.id=393f9d34e4534a2ebc37217c9e6a4ea9&scope.project.id=a89544d6d9d14e759c077d7175365dca&role.id=2b63bb896abc4a22ba92b589f8ba2a73 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:49.190147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9286c-256e-4b9a-bd33-b12380929743 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:49.208315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd069f-6cbc-4e18-b416-5ee62064a147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:49.209196 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd069f-6cbc-4e18-b416-5ee62064a147 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:49.209444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd069f-6cbc-4e18-b416-5ee62064a147 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:49.209723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd069f-6cbc-4e18-b416-5ee62064a147 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:49.233787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd069f-6cbc-4e18-b416-5ee62064a147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:49.249937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-dabd069f-6cbc-4e18-b416-5ee62064a147 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:49.249937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:49.249937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:49.249937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:49.249937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:49.249937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:49.249937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:49.249937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:49.249937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:49.249937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:49.249937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:49.249937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:49.249937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:49.249937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:49.249937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:49.249937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:49.251091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:49.251091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:49.251091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:49.251091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:49.251091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:49.251091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:49.251091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:49.251091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:49.251091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:49.251091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:49.251091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:49.251091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:49.251091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:49.251091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:49.251091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:49.251091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:49.252137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:49.252137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:49.252137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:49.252137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:49.252137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:49.252137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:49.252137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:49.252137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:49.252137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:49.252137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:49.252137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:49.252137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:31:49.252137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 131/251] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:49.268685 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:49.269973 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd069f-6cbc-4e18-b416-5ee62064a147 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:49.329455 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9286c-256e-4b9a-bd33-b12380929743 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:49.372822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2146d-3a17-4f2b-85c4-695246bc713f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:49.374042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2146d-3a17-4f2b-85c4-695246bc713f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:49.374670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2146d-3a17-4f2b-85c4-695246bc713f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:49.375108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2146d-3a17-4f2b-85c4-695246bc713f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:49.388388 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2146d-3a17-4f2b-85c4-695246bc713f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:49.397596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-94e2146d-3a17-4f2b-85c4-695246bc713f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:31:49.397596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:49.397596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:49.397596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:49.397596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:49.397596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:49.397596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:49.397596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:49.397596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:49.397596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:49.397596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:49.397596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:49.397596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:49.397596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:49.397596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:49.397596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:49.398570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:49.398570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:49.398570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:49.398570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:49.398570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:49.398570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:49.398570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:49.398570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:49.398570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:49.398570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:49.398570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:49.398570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:49.398570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:49.398570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:49.398570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:49.398570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:49.399518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:49.399518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:49.399518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:49.399518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:49.399518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:49.399518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:49.399518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:49.399518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:49.399518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:49.399518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:49.399518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:31:49.399518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:49.399518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 121/252] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:31:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:49.400677 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a89c46-6807-4b31-8054-efb9d700f547 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:49.400837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a89c46-6807-4b31-8054-efb9d700f547 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:49.402749 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a89c46-6807-4b31-8054-efb9d700f547 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:49.402749 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 122/253] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:49.408604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afaeb84-5f70-40ed-a66d-64b56d08a733 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:49.408929 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afaeb84-5f70-40ed-a66d-64b56d08a733 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:49.409174 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afaeb84-5f70-40ed-a66d-64b56d08a733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:49.429517 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-0afaeb84-5f70-40ed-a66d-64b56d08a733 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:49.430784 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-0afaeb84-5f70-40ed-a66d-64b56d08a733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:49.490061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd069f-6cbc-4e18-b416-5ee62064a147 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:49.514510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 123/254] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:49.516247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afaeb84-5f70-40ed-a66d-64b56d08a733 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:49.551433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272495a-d4d8-419a-8e46-4d3cb9315f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:49.552790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272495a-d4d8-419a-8e46-4d3cb9315f4b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:49.553086 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272495a-d4d8-419a-8e46-4d3cb9315f4b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:49.553435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272495a-d4d8-419a-8e46-4d3cb9315f4b None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:49.554266 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272495a-d4d8-419a-8e46-4d3cb9315f4b None admin] RBAC: Adding query filter params (name=member) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:49.554584 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272495a-d4d8-419a-8e46-4d3cb9315f4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:49.558606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272495a-d4d8-419a-8e46-4d3cb9315f4b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:49.568150 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:49.574416 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 132/255] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:49.580076 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272495a-d4d8-419a-8e46-4d3cb9315f4b None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:49.635400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:49.638430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afaeb84-5f70-40ed-a66d-64b56d08a733 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:49.680501 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc811e-74ae-47b5-95ae-52d337beb7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:49.681662 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc811e-74ae-47b5-95ae-52d337beb7ac None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:49.681968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc811e-74ae-47b5-95ae-52d337beb7ac None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:49.682257 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc811e-74ae-47b5-95ae-52d337beb7ac None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:49.683171 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc811e-74ae-47b5-95ae-52d337beb7ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:49.683652 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc811e-74ae-47b5-95ae-52d337beb7ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:49.685319 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc811e-74ae-47b5-95ae-52d337beb7ac None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:49.697502 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 124/256] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:49.698463 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc811e-74ae-47b5-95ae-52d337beb7ac None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:49.704828 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272495a-d4d8-419a-8e46-4d3cb9315f4b None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:49.777242 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7b87a-cc5c-4272-b7ee-174763db4d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:49.778502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7b87a-cc5c-4272-b7ee-174763db4d8e None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:49.778986 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7b87a-cc5c-4272-b7ee-174763db4d8e None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:49.779482 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7b87a-cc5c-4272-b7ee-174763db4d8e None admin] PATH_INFO: `/v3/users/b0232b4d01e84bb1a0a401bec86e975c` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:49.781887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7b87a-cc5c-4272-b7ee-174763db4d8e None admin] RBAC: Authorizing `identity:get_user(user_id=b0232b4d01e84bb1a0a401bec86e975c)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:49.783662 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7b87a-cc5c-4272-b7ee-174763db4d8e None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:49.791454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 133/257] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:49 2022] GET /identity/v3/users/b0232b4d01e84bb1a0a401bec86e975c => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:31:49.793949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7b87a-cc5c-4272-b7ee-174763db4d8e None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:49.915599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc811e-74ae-47b5-95ae-52d337beb7ac None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:49.921192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:49.994859 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7b87a-cc5c-4272-b7ee-174763db4d8e None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:50.009576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b0c863-eb39-4c2a-b053-f9cd8890a7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:50.010898 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0c863-eb39-4c2a-b053-f9cd8890a7be None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:50.011405 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0c863-eb39-4c2a-b053-f9cd8890a7be None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:50.011911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0c863-eb39-4c2a-b053-f9cd8890a7be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:50.018249 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:50.043364 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b0c863-eb39-4c2a-b053-f9cd8890a7be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:50.052186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-26b0c863-eb39-4c2a-b053-f9cd8890a7be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:31:50.052186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:50.052186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:50.052186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:50.052186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:50.052186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:50.052186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:50.052186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:50.052186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:50.052186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:50.052186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:50.052186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:50.052186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:50.052186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:50.052186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:50.052186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:50.053085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:50.053085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:50.053085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:50.053085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:50.053085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:50.053085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:50.053085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:50.053085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:50.053085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:50.053085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:50.053085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:50.053085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:50.053085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:50.053085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:50.053085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:50.053085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:50.054325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:50.054325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:50.054325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:50.054325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:50.054325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:50.054325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:50.054325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:50.054325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:50.054325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:50.054325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:50.054325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:31:50.054325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:50.054325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 125/258] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:31:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:50.054325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b0c863-eb39-4c2a-b053-f9cd8890a7be None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:50.071317 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf1293-f1e3-4cd6-9a0e-5731821fce63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:50.072249 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf1293-f1e3-4cd6-9a0e-5731821fce63 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:50.072537 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf1293-f1e3-4cd6-9a0e-5731821fce63 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:50.072813 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf1293-f1e3-4cd6-9a0e-5731821fce63 None admin] PATH_INFO: `/v3/users/e473cc17a9b64406b1aa1dac6b9ff5db` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:50.076825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf1293-f1e3-4cd6-9a0e-5731821fce63 None admin] RBAC: Authorizing `identity:get_user(user_id=e473cc17a9b64406b1aa1dac6b9ff5db)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:50.078296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf1293-f1e3-4cd6-9a0e-5731821fce63 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:50.080209 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 134/259] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:49 2022] GET /identity/v3/users/e473cc17a9b64406b1aa1dac6b9ff5db => generated 283 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:50.080812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf1293-f1e3-4cd6-9a0e-5731821fce63 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:50.249119 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf1293-f1e3-4cd6-9a0e-5731821fce63 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:50.299415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca448dc4-880a-4e9b-b57d-2cba024fe3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:50.300094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca448dc4-880a-4e9b-b57d-2cba024fe3cc None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:50.300384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca448dc4-880a-4e9b-b57d-2cba024fe3cc None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:50.300662 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca448dc4-880a-4e9b-b57d-2cba024fe3cc None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:50.301421 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca448dc4-880a-4e9b-b57d-2cba024fe3cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:50.302301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca448dc4-880a-4e9b-b57d-2cba024fe3cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:50.303705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca448dc4-880a-4e9b-b57d-2cba024fe3cc None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:50.318569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b0c863-eb39-4c2a-b053-f9cd8890a7be None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:50.323743 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 135/260] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:31:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:50.324402 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca448dc4-880a-4e9b-b57d-2cba024fe3cc None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:50.391341 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b7459-c328-42e3-b6fb-991923b11498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:50.393118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b7459-c328-42e3-b6fb-991923b11498 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:50.393830 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b7459-c328-42e3-b6fb-991923b11498 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:50.394368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b7459-c328-42e3-b6fb-991923b11498 None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:50.396643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b7459-c328-42e3-b6fb-991923b11498 None admin] RBAC: Authorizing `identity:get_project(project_id=a89544d6d9d14e759c077d7175365dca)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:50.399915 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b7459-c328-42e3-b6fb-991923b11498 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:50.404387 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 126/261] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:50 2022] GET /identity/v3/projects/a89544d6d9d14e759c077d7175365dca => generated 302 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:50.405459 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b7459-c328-42e3-b6fb-991923b11498 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:50.415701 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:50.477052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:50.523014 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca448dc4-880a-4e9b-b57d-2cba024fe3cc None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:50.571189 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b7459-c328-42e3-b6fb-991923b11498 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:50.593587 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f5aee-7ae0-414b-b321-d49f0c368353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:50.593940 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f5aee-7ae0-414b-b321-d49f0c368353 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:50.594209 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f5aee-7ae0-414b-b321-d49f0c368353 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:50.594582 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f5aee-7ae0-414b-b321-d49f0c368353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:50.618983 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd0806-edba-4af3-9343-7f8036ef30eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:50.620513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd0806-edba-4af3-9343-7f8036ef30eb None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:50.621363 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd0806-edba-4af3-9343-7f8036ef30eb None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:50.622475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd0806-edba-4af3-9343-7f8036ef30eb None admin] PATH_INFO: `/v3/users/e473cc17a9b64406b1aa1dac6b9ff5db` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:50.625574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd0806-edba-4af3-9343-7f8036ef30eb None admin] RBAC: Authorizing `identity:get_user(user_id=e473cc17a9b64406b1aa1dac6b9ff5db)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:50.629187 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd0806-edba-4af3-9343-7f8036ef30eb None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:50.633058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 127/262] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:50 2022] GET /identity/v3/users/e473cc17a9b64406b1aa1dac6b9ff5db => generated 283 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:50.634363 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd0806-edba-4af3-9343-7f8036ef30eb None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:50.741493 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:50.744935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd0806-edba-4af3-9343-7f8036ef30eb None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:50.761922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f5aee-7ae0-414b-b321-d49f0c368353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:50.763449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f5aee-7ae0-414b-b321-d49f0c368353 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:50.784362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd81fe-a432-436b-8cde-fcaab39ea01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:50.785645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd81fe-a432-436b-8cde-fcaab39ea01e None admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:50.786706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd81fe-a432-436b-8cde-fcaab39ea01e None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:50.787186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd81fe-a432-436b-8cde-fcaab39ea01e None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca/users/b0232b4d01e84bb1a0a401bec86e975c/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:50.790156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd81fe-a432-436b-8cde-fcaab39ea01e None admin] RBAC: Authorizing `identity:create_grant(project_id=a89544d6d9d14e759c077d7175365dca, user_id=b0232b4d01e84bb1a0a401bec86e975c, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:50.791935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd81fe-a432-436b-8cde-fcaab39ea01e None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:50.809695 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 128/263] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:31:50 2022] PUT /identity/v3/projects/a89544d6d9d14e759c077d7175365dca/users/b0232b4d01e84bb1a0a401bec86e975c/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:31:50.811263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd81fe-a432-436b-8cde-fcaab39ea01e None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:50.847811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:50.948293 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 136/264] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:31:50.952807 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd81fe-a432-436b-8cde-fcaab39ea01e None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:50.957541 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f5aee-7ae0-414b-b321-d49f0c368353 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:51.027640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb7edd-7097-424c-b147-7bd1e9b33997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:51.030010 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb7edd-7097-424c-b147-7bd1e9b33997 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:51.030790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb7edd-7097-424c-b147-7bd1e9b33997 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:51.033608 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb7edd-7097-424c-b147-7bd1e9b33997 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:51.097648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:51.142622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f5aee-7ae0-414b-b321-d49f0c368353 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:51.185486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb7edd-7097-424c-b147-7bd1e9b33997 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:51.187128 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb7edd-7097-424c-b147-7bd1e9b33997 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:51.212128 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155829b7-25c1-44f4-8107-6321e62b0950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:51.213457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155829b7-25c1-44f4-8107-6321e62b0950 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:51.214039 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155829b7-25c1-44f4-8107-6321e62b0950 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:51.214575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155829b7-25c1-44f4-8107-6321e62b0950 None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:51.216125 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155829b7-25c1-44f4-8107-6321e62b0950 None admin] RBAC: Authorizing `identity:get_project(project_id=a89544d6d9d14e759c077d7175365dca)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:51.217793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155829b7-25c1-44f4-8107-6321e62b0950 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:51.219794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 137/265] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:50 2022] GET /identity/v3/projects/a89544d6d9d14e759c077d7175365dca => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:51.221270 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bf066-b9b8-49a5-9c27-91d98843e2c7 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:51.221761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bf066-b9b8-49a5-9c27-91d98843e2c7 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:51.222225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bf066-b9b8-49a5-9c27-91d98843e2c7 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:51.224115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 138/266] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:51.229352 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296ca88-9ffd-4bad-920f-08756de0f377 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:51.229972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296ca88-9ffd-4bad-920f-08756de0f377 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:51.230439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296ca88-9ffd-4bad-920f-08756de0f377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:51.271448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-8296ca88-9ffd-4bad-920f-08756de0f377 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:51.273465 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-8296ca88-9ffd-4bad-920f-08756de0f377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:51.275940 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:51.304782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 129/267] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:31:51.305738 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb7edd-7097-424c-b147-7bd1e9b33997 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:51.396279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:51.420938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb7edd-7097-424c-b147-7bd1e9b33997 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:51.468497 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3462d42-d684-4513-b5ec-7e0d6c8a3f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:51.469163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3462d42-d684-4513-b5ec-7e0d6c8a3f76 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:51.469492 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3462d42-d684-4513-b5ec-7e0d6c8a3f76 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:51.469766 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3462d42-d684-4513-b5ec-7e0d6c8a3f76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:51.470667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3462d42-d684-4513-b5ec-7e0d6c8a3f76 None admin] RBAC: Adding query filter params (role.id=96c584428d924fdba9671949f760c406, scope.project.id=a89544d6d9d14e759c077d7175365dca, user.id=e473cc17a9b64406b1aa1dac6b9ff5db) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:51.470978 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3462d42-d684-4513-b5ec-7e0d6c8a3f76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:51.473095 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3462d42-d684-4513-b5ec-7e0d6c8a3f76 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:51.481401 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 130/268] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:31:51 2022] GET /identity/v3/role_assignments?user.id=e473cc17a9b64406b1aa1dac6b9ff5db&scope.project.id=a89544d6d9d14e759c077d7175365dca&role.id=96c584428d924fdba9671949f760c406 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:51.482388 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3462d42-d684-4513-b5ec-7e0d6c8a3f76 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:51.509209 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 139/269] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:51.511047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5469f8-876a-4dff-9212-2f5c67fd70b9 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:51.511189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5469f8-876a-4dff-9212-2f5c67fd70b9 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:51.511507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5469f8-876a-4dff-9212-2f5c67fd70b9 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:51.513030 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 140/270] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:51.521782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce5e38-a866-4662-9812-343e69c114b3 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:51.522118 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce5e38-a866-4662-9812-343e69c114b3 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:51.522347 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce5e38-a866-4662-9812-343e69c114b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:51.560538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-fbce5e38-a866-4662-9812-343e69c114b3 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:51.561747 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-fbce5e38-a866-4662-9812-343e69c114b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:51.635368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3462d42-d684-4513-b5ec-7e0d6c8a3f76 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:51.664894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:51.689443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be3963-6457-478f-bffc-0d8dad22d727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:51.690739 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be3963-6457-478f-bffc-0d8dad22d727 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:51.690847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 141/271] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:51.691396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be3963-6457-478f-bffc-0d8dad22d727 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:51.691686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce5e38-a866-4662-9812-343e69c114b3 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:51.692088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be3963-6457-478f-bffc-0d8dad22d727 None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:51.694242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be3963-6457-478f-bffc-0d8dad22d727 None admin] RBAC: Authorizing `identity:get_project(project_id=a89544d6d9d14e759c077d7175365dca)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:51.696100 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be3963-6457-478f-bffc-0d8dad22d727 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:51.698348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 131/272] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:51 2022] GET /identity/v3/projects/a89544d6d9d14e759c077d7175365dca => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:51.706083 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be3963-6457-478f-bffc-0d8dad22d727 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:51.819855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:51.886945 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce5e38-a866-4662-9812-343e69c114b3 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:51.899865 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be3963-6457-478f-bffc-0d8dad22d727 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:51.939456 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30726e4-b789-43f7-b6ab-1a6e9e8eebc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:51.940837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30726e4-b789-43f7-b6ab-1a6e9e8eebc4 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:51.941312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30726e4-b789-43f7-b6ab-1a6e9e8eebc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:51.941789 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30726e4-b789-43f7-b6ab-1a6e9e8eebc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:51.946798 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ec50e-b442-4644-85ce-863ce051eefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:51.948198 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ec50e-b442-4644-85ce-863ce051eefd None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:51.948802 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ec50e-b442-4644-85ce-863ce051eefd None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:51.948802 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ec50e-b442-4644-85ce-863ce051eefd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:51.949654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ec50e-b442-4644-85ce-863ce051eefd None admin] RBAC: Adding query filter params (role.id=b4be7021e4104740b8b0b3b40ecd3d59, scope.project.id=a89544d6d9d14e759c077d7175365dca, user.id=e473cc17a9b64406b1aa1dac6b9ff5db) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:51.950026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ec50e-b442-4644-85ce-863ce051eefd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:51.951536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ec50e-b442-4644-85ce-863ce051eefd None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:51.962843 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30726e4-b789-43f7-b6ab-1a6e9e8eebc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:51.963683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 132/273] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:31:51 2022] GET /identity/v3/role_assignments?user.id=e473cc17a9b64406b1aa1dac6b9ff5db&scope.project.id=a89544d6d9d14e759c077d7175365dca&role.id=b4be7021e4104740b8b0b3b40ecd3d59 => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:51.964668 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ec50e-b442-4644-85ce-863ce051eefd None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:51.971245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-e30726e4-b789-43f7-b6ab-1a6e9e8eebc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:51.971245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:51.971245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:51.971245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:51.971245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:51.971245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:51.971245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:51.971245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:51.971245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:51.971245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:51.971245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:51.971245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:51.971245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:51.971245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:51.971245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:51.971245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:51.972255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:51.972255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:51.972255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:51.972255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:51.972255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:51.972255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:51.972255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:51.972255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:51.972255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:51.972255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:51.972255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:51.972255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:51.972255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:51.972255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:51.972255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:51.972255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:51.973553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:51.973553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:51.973553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:51.973553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:51.973553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:51.973553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:51.973553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:51.973553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:51.973553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:51.973553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:51.973553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:51.973553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:31:51.973553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 142/274] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:51.975527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174ee8d-dd98-4ea3-84d4-fbcc81d0fbfc None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:51.975818 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174ee8d-dd98-4ea3-84d4-fbcc81d0fbfc None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:51.976095 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174ee8d-dd98-4ea3-84d4-fbcc81d0fbfc None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:51.977476 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 143/275] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:51.983834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba9089-6ffe-4ec6-aa4e-c14e2880520c None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:51.984141 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba9089-6ffe-4ec6-aa4e-c14e2880520c None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:51.984387 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba9089-6ffe-4ec6-aa4e-c14e2880520c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:52.009180 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-aaba9089-6ffe-4ec6-aa4e-c14e2880520c None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:52.010434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-aaba9089-6ffe-4ec6-aa4e-c14e2880520c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:52.048898 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:52.099191 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 144/276] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 22 11:31:52.100525 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba9089-6ffe-4ec6-aa4e-c14e2880520c None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:52.102025 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ec50e-b442-4644-85ce-863ce051eefd None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:52.171131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258e33b-7656-4c94-99c0-78332411c2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:52.172432 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258e33b-7656-4c94-99c0-78332411c2fe None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:52.172825 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258e33b-7656-4c94-99c0-78332411c2fe None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:52.173212 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258e33b-7656-4c94-99c0-78332411c2fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:52.175555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258e33b-7656-4c94-99c0-78332411c2fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:52.177842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258e33b-7656-4c94-99c0-78332411c2fe None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:52.180457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 133/277] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:31:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:52.182560 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258e33b-7656-4c94-99c0-78332411c2fe None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:52.190235 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:52.247035 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba9089-6ffe-4ec6-aa4e-c14e2880520c None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:52.317997 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258e33b-7656-4c94-99c0-78332411c2fe None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:52.325216 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c1e11-79ae-4054-895a-8b91596a09fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:52.326616 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c1e11-79ae-4054-895a-8b91596a09fa None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:52.326916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c1e11-79ae-4054-895a-8b91596a09fa None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:52.327196 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c1e11-79ae-4054-895a-8b91596a09fa None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:52.328028 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c1e11-79ae-4054-895a-8b91596a09fa None admin] RBAC: Adding query filter params (name=member) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:52.328341 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c1e11-79ae-4054-895a-8b91596a09fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:52.329834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c1e11-79ae-4054-895a-8b91596a09fa None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:52.339790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 145/278] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:52.340493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c1e11-79ae-4054-895a-8b91596a09fa None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:52.349861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:52.366302 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb8320-367d-4381-9f01-8eccf1ef2deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:52.367174 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb8320-367d-4381-9f01-8eccf1ef2deb None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:52.367603 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb8320-367d-4381-9f01-8eccf1ef2deb None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:52.367889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb8320-367d-4381-9f01-8eccf1ef2deb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:52.383341 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb8320-367d-4381-9f01-8eccf1ef2deb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:52.391857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-19eb8320-367d-4381-9f01-8eccf1ef2deb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 22 11:31:52.391857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:52.391857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:52.391857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:52.391857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:52.391857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:52.391857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:52.391857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:52.391857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:52.391857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:52.391857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:52.391857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:52.391857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:52.391857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:52.391857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:52.391857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:52.393011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:52.393011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:52.393011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:52.393011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:52.393011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:52.393011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:52.393011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:52.393011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:52.393011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:52.393011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:52.393011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:52.393011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:52.393011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:52.393011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:52.393011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:52.393011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:52.394130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:52.394130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:52.394130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:52.394130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:52.394130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:52.394130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:52.394130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:52.394130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:52.394130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:52.394130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:52.394130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 22 11:31:52.394130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:52.398532 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 134/279] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:52.399950 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb8320-367d-4381-9f01-8eccf1ef2deb None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:52.491613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c1e11-79ae-4054-895a-8b91596a09fa None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:52.518639 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:52.538070 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c782dd-8804-4743-905b-7777a05b05c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:52.538771 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c782dd-8804-4743-905b-7777a05b05c7 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:52.539059 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c782dd-8804-4743-905b-7777a05b05c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:52.539324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c782dd-8804-4743-905b-7777a05b05c7 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:52.540084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c782dd-8804-4743-905b-7777a05b05c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:52.541581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c782dd-8804-4743-905b-7777a05b05c7 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:52.545299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb8320-367d-4381-9f01-8eccf1ef2deb None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:52.568476 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 146/280] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jun 22 11:31:52 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:31:52.568476 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c782dd-8804-4743-905b-7777a05b05c7 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:52.603581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385c9c8-a95b-4a57-9879-95ca099b271a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:52.604814 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385c9c8-a95b-4a57-9879-95ca099b271a None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:52.605173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385c9c8-a95b-4a57-9879-95ca099b271a None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:52.605552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385c9c8-a95b-4a57-9879-95ca099b271a None admin] PATH_INFO: `/v3/users/a58210d2064e416fb037849f65f8b617` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:52.633890 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385c9c8-a95b-4a57-9879-95ca099b271a None admin] RBAC: Authorizing `identity:get_user(user_id=a58210d2064e416fb037849f65f8b617)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:52.635503 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385c9c8-a95b-4a57-9879-95ca099b271a None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:52.643117 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 135/281] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:52 2022] GET /identity/v3/users/a58210d2064e416fb037849f65f8b617 => generated 293 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:52.648640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385c9c8-a95b-4a57-9879-95ca099b271a None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:52.729007 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:52.769714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c782dd-8804-4743-905b-7777a05b05c7 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:52.824659 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385c9c8-a95b-4a57-9879-95ca099b271a None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:52.830584 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078101c6-e69a-4f92-9dc3-32624f24e365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:52.831453 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078101c6-e69a-4f92-9dc3-32624f24e365 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:52.831741 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078101c6-e69a-4f92-9dc3-32624f24e365 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:52.832047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078101c6-e69a-4f92-9dc3-32624f24e365 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:52.832857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078101c6-e69a-4f92-9dc3-32624f24e365 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:52.833264 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078101c6-e69a-4f92-9dc3-32624f24e365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:52.834740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078101c6-e69a-4f92-9dc3-32624f24e365 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:52.848714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 147/282] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:52.856868 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078101c6-e69a-4f92-9dc3-32624f24e365 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:52.874082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774909da-8bba-4689-aea2-ccc6886c26be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:52.874877 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774909da-8bba-4689-aea2-ccc6886c26be None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:52.875239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774909da-8bba-4689-aea2-ccc6886c26be None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:52.875621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774909da-8bba-4689-aea2-ccc6886c26be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:52.876566 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774909da-8bba-4689-aea2-ccc6886c26be None admin] RBAC: Adding query filter params (role.id=f2336919c5104f6a8210e99994f4d455, user.id=a58210d2064e416fb037849f65f8b617, scope.system=all) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:52.877009 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774909da-8bba-4689-aea2-ccc6886c26be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:52.878577 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774909da-8bba-4689-aea2-ccc6886c26be None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:52.885677 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 136/283] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Jun 22 11:31:52 2022] GET /identity/v3/role_assignments?user.id=a58210d2064e416fb037849f65f8b617&scope.system=all&role.id=f2336919c5104f6a8210e99994f4d455 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:31:52.918670 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:53.013119 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078101c6-e69a-4f92-9dc3-32624f24e365 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:53.060830 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3acd1-7762-4a37-9260-31e7f571fe17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:53.061696 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3acd1-7762-4a37-9260-31e7f571fe17 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:53.061974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3acd1-7762-4a37-9260-31e7f571fe17 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:53.062249 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3acd1-7762-4a37-9260-31e7f571fe17 None admin] PATH_INFO: `/v3/users/393f9d34e4534a2ebc37217c9e6a4ea9` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:53.064540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f3acd1-7762-4a37-9260-31e7f571fe17 None admin] RBAC: Authorizing `identity:get_user(user_id=393f9d34e4534a2ebc37217c9e6a4ea9)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:53.066611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f3acd1-7762-4a37-9260-31e7f571fe17 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:53.070644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 148/284] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:52 2022] GET /identity/v3/users/393f9d34e4534a2ebc37217c9e6a4ea9 => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:53.076411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774909da-8bba-4689-aea2-ccc6886c26be None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:53.259334 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774909da-8bba-4689-aea2-ccc6886c26be None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:53.281549 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:53.332764 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170bc069-c732-4691-82d6-3d047c888a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:53.333900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bc069-c732-4691-82d6-3d047c888a8a None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:53.334326 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bc069-c732-4691-82d6-3d047c888a8a None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:53.334752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170bc069-c732-4691-82d6-3d047c888a8a None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:53.336359 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170bc069-c732-4691-82d6-3d047c888a8a None admin] RBAC: Authorizing `identity:get_project(project_id=a89544d6d9d14e759c077d7175365dca)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:53.336489 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832eb59a-89ab-437a-a346-04ba8d1a4d77 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:53.337090 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832eb59a-89ab-437a-a346-04ba8d1a4d77 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:53.337090 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832eb59a-89ab-437a-a346-04ba8d1a4d77 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:53.338649 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 149/285] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:53.340246 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170bc069-c732-4691-82d6-3d047c888a8a None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:53.343821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 137/286] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:53 2022] GET /identity/v3/projects/a89544d6d9d14e759c077d7175365dca => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:53.350689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832eb59a-89ab-437a-a346-04ba8d1a4d77 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:53.357126 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52863029-7f05-44fe-bc7f-abbee1a1ed9f None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:53.358305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52863029-7f05-44fe-bc7f-abbee1a1ed9f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:53.360773 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52863029-7f05-44fe-bc7f-abbee1a1ed9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:53.400485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-52863029-7f05-44fe-bc7f-abbee1a1ed9f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:53.401743 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-52863029-7f05-44fe-bc7f-abbee1a1ed9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:53.504887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832eb59a-89ab-437a-a346-04ba8d1a4d77 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:53.522536 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:53.577749 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 138/287] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:53.607954 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a7986-f2a3-44ca-bfaa-868d836d622a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:53.609418 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a7986-f2a3-44ca-bfaa-868d836d622a None admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:53.609929 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a7986-f2a3-44ca-bfaa-868d836d622a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:53.610413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a7986-f2a3-44ca-bfaa-868d836d622a None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca/users/393f9d34e4534a2ebc37217c9e6a4ea9/roles/2b63bb896abc4a22ba92b589f8ba2a73` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:53.615130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a7986-f2a3-44ca-bfaa-868d836d622a None admin] RBAC: Authorizing `identity:create_grant(project_id=a89544d6d9d14e759c077d7175365dca, user_id=393f9d34e4534a2ebc37217c9e6a4ea9, role_id=2b63bb896abc4a22ba92b589f8ba2a73)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:53.617560 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a7986-f2a3-44ca-bfaa-868d836d622a None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:53.624699 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52863029-7f05-44fe-bc7f-abbee1a1ed9f None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:53.647026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 150/288] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:31:53 2022] PUT /identity/v3/projects/a89544d6d9d14e759c077d7175365dca/users/393f9d34e4534a2ebc37217c9e6a4ea9/roles/2b63bb896abc4a22ba92b589f8ba2a73 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:31:53.797419 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52863029-7f05-44fe-bc7f-abbee1a1ed9f None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:53.801911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:53.850623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c2881-2563-473f-ba85-7eb8677d1a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:53.852398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c2881-2563-473f-ba85-7eb8677d1a51 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:53.852398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c2881-2563-473f-ba85-7eb8677d1a51 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:53.852398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c2881-2563-473f-ba85-7eb8677d1a51 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:53.863975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c2881-2563-473f-ba85-7eb8677d1a51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:53.872042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-414c2881-2563-473f-ba85-7eb8677d1a51 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:53.872042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:53.872042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:53.872042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:53.872042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:53.872042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:53.872042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:53.872042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:53.872042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:53.872042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:53.872042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:53.872042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:53.872042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:53.872042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:53.872042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:53.872042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:53.872969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:53.872969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:53.872969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:53.872969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:53.872969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:53.872969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:53.872969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:53.872969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:53.872969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:53.872969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:53.872969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:53.872969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:53.872969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:53.872969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:53.872969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:53.872969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:53.874197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:53.874197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:53.874197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:53.874197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:53.874197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:53.874197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:53.874197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:53.874197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:53.874197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:53.874197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:53.874197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:53.874197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:53.874197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 139/289] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:53.881784 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a7986-f2a3-44ca-bfaa-868d836d622a None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:54.000215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a7986-f2a3-44ca-bfaa-868d836d622a None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:54.039476 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb061e3-bcba-4afd-8e31-aed105a7eb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:54.040219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb061e3-bcba-4afd-8e31-aed105a7eb75 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:54.040505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb061e3-bcba-4afd-8e31-aed105a7eb75 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:54.040836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb061e3-bcba-4afd-8e31-aed105a7eb75 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:54.041655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb061e3-bcba-4afd-8e31-aed105a7eb75 None admin] RBAC: Adding query filter params (name=member) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:54.041970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb061e3-bcba-4afd-8e31-aed105a7eb75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:54.043388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb061e3-bcba-4afd-8e31-aed105a7eb75 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:54.053759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 151/290] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:54.059574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c2881-2563-473f-ba85-7eb8677d1a51 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:54.213811 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c2881-2563-473f-ba85-7eb8677d1a51 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:54.260563 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502e39b-b390-4407-9e55-31d815db281e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:54.261910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502e39b-b390-4407-9e55-31d815db281e None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:54.262483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502e39b-b390-4407-9e55-31d815db281e None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:54.263003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502e39b-b390-4407-9e55-31d815db281e None admin] PATH_INFO: `/v3/users/b0232b4d01e84bb1a0a401bec86e975c` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:54.268399 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502e39b-b390-4407-9e55-31d815db281e None admin] RBAC: Authorizing `identity:get_user(user_id=b0232b4d01e84bb1a0a401bec86e975c)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:54.270413 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502e39b-b390-4407-9e55-31d815db281e None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:54.272594 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 140/291] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:54 2022] GET /identity/v3/users/b0232b4d01e84bb1a0a401bec86e975c => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:54.280538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb061e3-bcba-4afd-8e31-aed105a7eb75 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:54.331062 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:54.456793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb061e3-bcba-4afd-8e31-aed105a7eb75 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:54.514609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922e331-78e8-4db8-8365-a7326bb4fc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:54.515992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922e331-78e8-4db8-8365-a7326bb4fc65 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:54.516501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922e331-78e8-4db8-8365-a7326bb4fc65 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:54.516993 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1922e331-78e8-4db8-8365-a7326bb4fc65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:54.612027 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922e331-78e8-4db8-8365-a7326bb4fc65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:54.615570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1922e331-78e8-4db8-8365-a7326bb4fc65 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:54.656939 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:54.739463 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 152/292] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:31:54.748515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502e39b-b390-4407-9e55-31d815db281e None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:54.810637 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:54.902260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502e39b-b390-4407-9e55-31d815db281e None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:54.956561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18a7c8-34ec-4eb6-8639-9aa2bef3ad4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:54.958625 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18a7c8-34ec-4eb6-8639-9aa2bef3ad4f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:54.959131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18a7c8-34ec-4eb6-8639-9aa2bef3ad4f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:54.959654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18a7c8-34ec-4eb6-8639-9aa2bef3ad4f None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:54.961896 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff18a7c8-34ec-4eb6-8639-9aa2bef3ad4f None admin] RBAC: Authorizing `identity:get_project(project_id=a89544d6d9d14e759c077d7175365dca)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:54.964655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff18a7c8-34ec-4eb6-8639-9aa2bef3ad4f None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:54.968489 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 141/293] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:54 2022] GET /identity/v3/projects/a89544d6d9d14e759c077d7175365dca => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:54.981238 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922e331-78e8-4db8-8365-a7326bb4fc65 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:55.269756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1922e331-78e8-4db8-8365-a7326bb4fc65 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:55.345696 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:55.368049 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b657d33-f34a-4c69-a7ca-4ef4328a1617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:55.369355 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b657d33-f34a-4c69-a7ca-4ef4328a1617 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:55.369860 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b657d33-f34a-4c69-a7ca-4ef4328a1617 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:55.370350 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b657d33-f34a-4c69-a7ca-4ef4328a1617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:55.372015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b657d33-f34a-4c69-a7ca-4ef4328a1617 None admin] RBAC: Adding query filter params (role.id=f2336919c5104f6a8210e99994f4d455, scope.project.id=a89544d6d9d14e759c077d7175365dca, user.id=b0232b4d01e84bb1a0a401bec86e975c) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:55.372586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b657d33-f34a-4c69-a7ca-4ef4328a1617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:55.380762 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b657d33-f34a-4c69-a7ca-4ef4328a1617 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:55.397659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 153/294] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:31:54 2022] GET /identity/v3/role_assignments?user.id=b0232b4d01e84bb1a0a401bec86e975c&scope.project.id=a89544d6d9d14e759c077d7175365dca&role.id=f2336919c5104f6a8210e99994f4d455 => generated 608 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:55.460311 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c2f06-d200-483c-bc9d-35bac4f7a6e6 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:55.460781 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c2f06-d200-483c-bc9d-35bac4f7a6e6 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:55.461247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c2f06-d200-483c-bc9d-35bac4f7a6e6 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:55.464031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 142/295] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:55.473068 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf2bee-b8dc-4a8e-9615-27d26b4951b8 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:55.474152 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf2bee-b8dc-4a8e-9615-27d26b4951b8 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:55.474608 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf2bee-b8dc-4a8e-9615-27d26b4951b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:55.519939 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-97cf2bee-b8dc-4a8e-9615-27d26b4951b8 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:55.521153 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-97cf2bee-b8dc-4a8e-9615-27d26b4951b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:55.545302 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d072e41-7f6c-4321-9fdf-3a033066dfaf None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:55.545834 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d072e41-7f6c-4321-9fdf-3a033066dfaf None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:55.546504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d072e41-7f6c-4321-9fdf-3a033066dfaf None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:55.549518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 143/296] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:55.557004 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514814d-40c7-427f-a37f-72bbbe710892 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:55.557502 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514814d-40c7-427f-a37f-72bbbe710892 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:55.557988 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514814d-40c7-427f-a37f-72bbbe710892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:55.600174 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-c514814d-40c7-427f-a37f-72bbbe710892 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:55.601398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-c514814d-40c7-427f-a37f-72bbbe710892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:55.656731 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 154/297] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:55.722304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf2bee-b8dc-4a8e-9615-27d26b4951b8 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:55.744171 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 144/298] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 22 11:31:55.769746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514814d-40c7-427f-a37f-72bbbe710892 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:55.805400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:55.884541 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514814d-40c7-427f-a37f-72bbbe710892 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:55.924242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd07d3f-6ba7-4048-bd54-b8e8c918f84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:55.925049 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd07d3f-6ba7-4048-bd54-b8e8c918f84e None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:55.925383 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd07d3f-6ba7-4048-bd54-b8e8c918f84e None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:55.925627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd07d3f-6ba7-4048-bd54-b8e8c918f84e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:55.930862 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf2bee-b8dc-4a8e-9615-27d26b4951b8 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:55.949609 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd07d3f-6ba7-4048-bd54-b8e8c918f84e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:55.958191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-bfd07d3f-6ba7-4048-bd54-b8e8c918f84e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 22 11:31:55.958191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:55.958191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:55.958191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:55.958191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:55.958191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:55.958191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:55.958191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:55.958191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:55.958191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:55.958191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:55.958191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:55.958191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:55.958191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:55.958191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:55.958191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:55.959123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:55.959123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:55.959123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:55.959123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:55.959123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:55.959123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:55.959123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:55.959123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:55.959123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:55.959123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:55.959123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:55.959123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:55.959123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:55.959123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:55.959123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:55.959123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:55.959962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:55.959962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:55.959962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:55.959962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:55.959962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:55.959962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:55.959962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:55.959962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:55.959962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:55.959962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:55.959962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 22 11:31:55.959962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:55.960556 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 145/299] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:55.966830 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd07d3f-6ba7-4048-bd54-b8e8c918f84e None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:55.981860 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:56.007091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86a166-52f6-47e6-9054-3288f2921b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:56.008240 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86a166-52f6-47e6-9054-3288f2921b00 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:56.008669 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86a166-52f6-47e6-9054-3288f2921b00 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:56.009084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86a166-52f6-47e6-9054-3288f2921b00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:56.012321 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86a166-52f6-47e6-9054-3288f2921b00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:56.014274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86a166-52f6-47e6-9054-3288f2921b00 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:56.023158 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 155/300] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:31:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:56.025334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86a166-52f6-47e6-9054-3288f2921b00 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:56.125960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:56.140900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd07d3f-6ba7-4048-bd54-b8e8c918f84e None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:56.152607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86a166-52f6-47e6-9054-3288f2921b00 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:56.186872 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba42316-526c-4fe4-ba0d-ade1999eeffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:56.187866 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba42316-526c-4fe4-ba0d-ade1999eeffa None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:56.188152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba42316-526c-4fe4-ba0d-ade1999eeffa None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:56.188429 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba42316-526c-4fe4-ba0d-ade1999eeffa None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:56.189194 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba42316-526c-4fe4-ba0d-ade1999eeffa None admin] RBAC: Adding query filter params (name=reader) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:56.189504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba42316-526c-4fe4-ba0d-ade1999eeffa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:56.191252 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba42316-526c-4fe4-ba0d-ade1999eeffa None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:56.203166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 146/301] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:56.209494 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba42316-526c-4fe4-ba0d-ade1999eeffa None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:56.215967 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538eba2-1666-4dc8-bd24-d32dd32758bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:56.217246 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538eba2-1666-4dc8-bd24-d32dd32758bf None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:56.217909 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538eba2-1666-4dc8-bd24-d32dd32758bf None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:56.218488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538eba2-1666-4dc8-bd24-d32dd32758bf None admin] PATH_INFO: `/v3/groups` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:56.220052 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538eba2-1666-4dc8-bd24-d32dd32758bf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:56.222951 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538eba2-1666-4dc8-bd24-d32dd32758bf None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:56.253764 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 156/302] 10.222.0.36 () {60 vars in 1134 bytes} [Wed Jun 22 11:31:56 2022] POST /identity/v3/groups => generated 229 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:31:56.306670 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b93d35-ae72-4a87-a481-e2e92142df23 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:56.307194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b93d35-ae72-4a87-a481-e2e92142df23 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:56.307790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b93d35-ae72-4a87-a481-e2e92142df23 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:56.315269 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 157/303] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:56.317715 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154d487-1497-42f9-af42-52ad9cc7ff16 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:56.318010 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154d487-1497-42f9-af42-52ad9cc7ff16 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:56.318292 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c154d487-1497-42f9-af42-52ad9cc7ff16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:56.338326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c154d487-1497-42f9-af42-52ad9cc7ff16 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:56.339663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c154d487-1497-42f9-af42-52ad9cc7ff16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:56.365220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba42316-526c-4fe4-ba0d-ade1999eeffa None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:56.399677 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:56.408537 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c8247-1924-4d00-a24e-5dec14d2bc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:56.409805 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c8247-1924-4d00-a24e-5dec14d2bc3a None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:56.410322 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c8247-1924-4d00-a24e-5dec14d2bc3a None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:56.410875 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c8247-1924-4d00-a24e-5dec14d2bc3a None admin] PATH_INFO: `/v3/users/8d351384dae546b5a750bba731b9cd09` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:56.426558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c8247-1924-4d00-a24e-5dec14d2bc3a None admin] RBAC: Authorizing `identity:get_user(user_id=8d351384dae546b5a750bba731b9cd09)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:56.431395 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 158/304] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:56.435772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c8247-1924-4d00-a24e-5dec14d2bc3a None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:56.437928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 147/305] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:56 2022] GET /identity/v3/users/8d351384dae546b5a750bba731b9cd09 => generated 293 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:56.448593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154d487-1497-42f9-af42-52ad9cc7ff16 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:56.462803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c8247-1924-4d00-a24e-5dec14d2bc3a None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:56.478496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:56.608209 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c154d487-1497-42f9-af42-52ad9cc7ff16 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:56.610273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c8247-1924-4d00-a24e-5dec14d2bc3a None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:56.647372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147573f5-2774-47c2-8dad-b46aa48bf08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:56.648092 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147573f5-2774-47c2-8dad-b46aa48bf08d None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:56.648381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147573f5-2774-47c2-8dad-b46aa48bf08d None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:56.648660 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147573f5-2774-47c2-8dad-b46aa48bf08d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:56.649499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147573f5-2774-47c2-8dad-b46aa48bf08d None admin] RBAC: Adding query filter params (role.id=2b63bb896abc4a22ba92b589f8ba2a73, user.id=8d351384dae546b5a750bba731b9cd09, scope.system=all) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:56.649863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147573f5-2774-47c2-8dad-b46aa48bf08d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:56.651262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147573f5-2774-47c2-8dad-b46aa48bf08d None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:56.657372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 159/306] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Jun 22 11:31:56 2022] GET /identity/v3/role_assignments?user.id=8d351384dae546b5a750bba731b9cd09&scope.system=all&role.id=2b63bb896abc4a22ba92b589f8ba2a73 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:56.657539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28107cd-9255-4549-ade4-54605ad4730d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:56.658241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28107cd-9255-4549-ade4-54605ad4730d None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:56.658535 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28107cd-9255-4549-ade4-54605ad4730d None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:56.658815 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28107cd-9255-4549-ade4-54605ad4730d None admin] PATH_INFO: `/v3/roles/member` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:56.672097 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28107cd-9255-4549-ade4-54605ad4730d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:56.680715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-c28107cd-9255-4549-ade4-54605ad4730d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:56.680715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:56.680715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:56.680715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:56.680715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:56.680715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:56.680715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:56.680715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:56.680715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:56.680715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:56.680715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:56.680715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:56.680715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:56.680715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:56.680715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:56.680715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:56.681686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:56.681686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:56.681686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:56.681686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:56.681686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:56.681686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:56.681686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:56.681686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:56.681686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:56.681686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:56.681686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:56.681686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:56.681686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:56.681686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:56.681686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:56.681686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:56.682476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:56.682476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:56.682476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:56.682476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:56.682476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:56.682476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:56.682476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:56.682476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:56.682476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:56.682476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:56.682476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:56.682476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:56.682476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 148/307] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:56.689960 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147573f5-2774-47c2-8dad-b46aa48bf08d None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:56.711132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfebaf-e644-4854-9dac-55c7628a0152 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:56.711871 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfebaf-e644-4854-9dac-55c7628a0152 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:56.712151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cfebaf-e644-4854-9dac-55c7628a0152 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:56.713940 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 149/308] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:31:56.721541 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c378ef-e952-4f38-9faf-8579e61c9dad None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:56.721975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c378ef-e952-4f38-9faf-8579e61c9dad None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:56.722353 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c378ef-e952-4f38-9faf-8579e61c9dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:56.748847 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-32c378ef-e952-4f38-9faf-8579e61c9dad None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:56.750356 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-32c378ef-e952-4f38-9faf-8579e61c9dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:56.773733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:56.835065 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147573f5-2774-47c2-8dad-b46aa48bf08d None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:56.845131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:56.855768 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 150/309] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:56.876151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c378ef-e952-4f38-9faf-8579e61c9dad None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:56.876597 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f8bae-e8fd-4c3a-b499-cc8186ec1df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:56.877278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f8bae-e8fd-4c3a-b499-cc8186ec1df1 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:56.877556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f8bae-e8fd-4c3a-b499-cc8186ec1df1 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:56.877828 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f8bae-e8fd-4c3a-b499-cc8186ec1df1 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:56.878584 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f8bae-e8fd-4c3a-b499-cc8186ec1df1 None admin] RBAC: Adding query filter params (name=member) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:56.878888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f8bae-e8fd-4c3a-b499-cc8186ec1df1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:56.881475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0f8bae-e8fd-4c3a-b499-cc8186ec1df1 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:56.893017 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 160/310] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:56.898608 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f8bae-e8fd-4c3a-b499-cc8186ec1df1 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:56.999534 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c378ef-e952-4f38-9faf-8579e61c9dad None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:57.040127 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0f8bae-e8fd-4c3a-b499-cc8186ec1df1 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:57.044055 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf04289-798c-494d-ac0d-ea409cb68840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:57.044344 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf04289-798c-494d-ac0d-ea409cb68840 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:57.044546 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf04289-798c-494d-ac0d-ea409cb68840 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:57.044818 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf04289-798c-494d-ac0d-ea409cb68840 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:57.050506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:57.065467 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf04289-798c-494d-ac0d-ea409cb68840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:57.077239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-ccf04289-798c-494d-ac0d-ea409cb68840 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 22 11:31:57.077239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:57.077239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:57.077239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:57.077239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:57.077239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:57.077239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:57.077239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:57.077239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:57.077239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:57.077239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:57.077239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:57.077239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:57.077239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:57.077239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:57.077239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:57.078188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:57.078188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:57.078188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:57.078188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:57.078188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:57.078188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:57.078188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:57.078188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:57.078188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:57.078188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:57.078188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:57.078188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:57.078188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:57.078188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:57.078188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:57.078188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:57.078970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:57.078970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:57.078970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:57.078970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:57.078970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:57.078970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:57.078970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:57.078970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:57.078970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:57.078970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:57.078970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 22 11:31:57.078970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:57.086019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 151/311] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:57.088213 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf04289-798c-494d-ac0d-ea409cb68840 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:57.097484 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfa302-f1f2-4d4f-8ce8-b53d7b88e781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:57.100194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfa302-f1f2-4d4f-8ce8-b53d7b88e781 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:57.100717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfa302-f1f2-4d4f-8ce8-b53d7b88e781 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:57.101187 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfa302-f1f2-4d4f-8ce8-b53d7b88e781 None admin] PATH_INFO: `/v3/users/a58210d2064e416fb037849f65f8b617` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:57.103305 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bfa302-f1f2-4d4f-8ce8-b53d7b88e781 None admin] RBAC: Authorizing `identity:get_user(user_id=a58210d2064e416fb037849f65f8b617)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:57.105870 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bfa302-f1f2-4d4f-8ce8-b53d7b88e781 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:57.109242 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 161/312] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:56 2022] GET /identity/v3/users/a58210d2064e416fb037849f65f8b617 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:57.115007 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfa302-f1f2-4d4f-8ce8-b53d7b88e781 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:57.174208 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:57.232930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf04289-798c-494d-ac0d-ea409cb68840 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:57.235585 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfa302-f1f2-4d4f-8ce8-b53d7b88e781 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:57.272517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f9501-a19e-4fc6-8d04-d1c820dde4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:57.273496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f9501-a19e-4fc6-8d04-d1c820dde4a3 None admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:57.273779 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f9501-a19e-4fc6-8d04-d1c820dde4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:57.274056 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f9501-a19e-4fc6-8d04-d1c820dde4a3 None admin] PATH_INFO: `/v3/system/users/a58210d2064e416fb037849f65f8b617/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:57.275099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc482ee9-2a42-4d26-bab7-5d8e2c32357b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:57.275582 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f9501-a19e-4fc6-8d04-d1c820dde4a3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a58210d2064e416fb037849f65f8b617, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:57.276264 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc482ee9-2a42-4d26-bab7-5d8e2c32357b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:57.276739 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc482ee9-2a42-4d26-bab7-5d8e2c32357b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:57.276975 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f9501-a19e-4fc6-8d04-d1c820dde4a3 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:57.277424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc482ee9-2a42-4d26-bab7-5d8e2c32357b None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:57.278383 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc482ee9-2a42-4d26-bab7-5d8e2c32357b None admin] RBAC: Adding query filter params (name=reader) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:57.278882 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc482ee9-2a42-4d26-bab7-5d8e2c32357b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:57.280479 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc482ee9-2a42-4d26-bab7-5d8e2c32357b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:57.293544 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 152/314] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:57.298235 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 162/314] 10.222.0.36 () {58 vars in 1334 bytes} [Wed Jun 22 11:31:57 2022] PUT /identity/v3/system/users/a58210d2064e416fb037849f65f8b617/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:31:57.300991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f9501-a19e-4fc6-8d04-d1c820dde4a3 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:57.351826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:57.436854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f9501-a19e-4fc6-8d04-d1c820dde4a3 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:57.478976 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ff2ad-3091-47be-813d-62c6acea23c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:57.479952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ff2ad-3091-47be-813d-62c6acea23c0 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:57.480228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ff2ad-3091-47be-813d-62c6acea23c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:57.480556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ff2ad-3091-47be-813d-62c6acea23c0 None admin] PATH_INFO: `/v3/users/393f9d34e4534a2ebc37217c9e6a4ea9` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:57.481813 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ff2ad-3091-47be-813d-62c6acea23c0 None admin] RBAC: Authorizing `identity:get_user(user_id=393f9d34e4534a2ebc37217c9e6a4ea9)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:57.483313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ff2ad-3091-47be-813d-62c6acea23c0 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:57.485115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 163/315] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:57 2022] GET /identity/v3/users/393f9d34e4534a2ebc37217c9e6a4ea9 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:31:57.490721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc482ee9-2a42-4d26-bab7-5d8e2c32357b None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:57.607292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc482ee9-2a42-4d26-bab7-5d8e2c32357b None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:57.632091 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:57.646492 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba6c93-005f-49c8-b6dd-4de21c8a62ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:57.647154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba6c93-005f-49c8-b6dd-4de21c8a62ba None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:57.647464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba6c93-005f-49c8-b6dd-4de21c8a62ba None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:57.647738 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba6c93-005f-49c8-b6dd-4de21c8a62ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:57.731406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba6c93-005f-49c8-b6dd-4de21c8a62ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:57.732875 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba6c93-005f-49c8-b6dd-4de21c8a62ba None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:57.850272 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 153/316] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:31:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:31:57.855408 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ff2ad-3091-47be-813d-62c6acea23c0 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:57.977456 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ff2ad-3091-47be-813d-62c6acea23c0 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:57.986381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:58.026344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcdd48-8915-4e37-9b6b-a741769ad3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:58.027525 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcdd48-8915-4e37-9b6b-a741769ad3fa None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:58.027965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcdd48-8915-4e37-9b6b-a741769ad3fa None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:58.028382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcdd48-8915-4e37-9b6b-a741769ad3fa None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:58.029769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcdd48-8915-4e37-9b6b-a741769ad3fa None admin] RBAC: Authorizing `identity:get_project(project_id=a89544d6d9d14e759c077d7175365dca)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:58.031389 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcdd48-8915-4e37-9b6b-a741769ad3fa None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:58.033573 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 164/317] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:31:57 2022] GET /identity/v3/projects/a89544d6d9d14e759c077d7175365dca => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:58.048546 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba6c93-005f-49c8-b6dd-4de21c8a62ba None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:58.081993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:58.217156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba6c93-005f-49c8-b6dd-4de21c8a62ba None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:58.269183 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae435ef-2d4a-4b18-8261-60ea63c13909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:58.269986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae435ef-2d4a-4b18-8261-60ea63c13909 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:58.270932 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae435ef-2d4a-4b18-8261-60ea63c13909 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:58.271463 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae435ef-2d4a-4b18-8261-60ea63c13909 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:58.272470 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae435ef-2d4a-4b18-8261-60ea63c13909 None admin] RBAC: Adding query filter params (role.id=2b63bb896abc4a22ba92b589f8ba2a73, scope.project.id=a89544d6d9d14e759c077d7175365dca, user.id=393f9d34e4534a2ebc37217c9e6a4ea9) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:58.272791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae435ef-2d4a-4b18-8261-60ea63c13909 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:58.274413 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae435ef-2d4a-4b18-8261-60ea63c13909 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:58.283265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 154/318] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:31:58 2022] GET /identity/v3/role_assignments?user.id=393f9d34e4534a2ebc37217c9e6a4ea9&scope.project.id=a89544d6d9d14e759c077d7175365dca&role.id=2b63bb896abc4a22ba92b589f8ba2a73 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:58.686547 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129110ed-2146-4d42-9960-becdda33d199 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:58.686814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129110ed-2146-4d42-9960-becdda33d199 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:58.687080 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129110ed-2146-4d42-9960-becdda33d199 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:58.688706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 165/319] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:31:58.695862 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e1b41-ccf8-41e0-8e66-7dc9263edde3 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:58.696499 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e1b41-ccf8-41e0-8e66-7dc9263edde3 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:58.697106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e1b41-ccf8-41e0-8e66-7dc9263edde3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:58.727466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-770e1b41-ccf8-41e0-8e66-7dc9263edde3 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:58.729603 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-770e1b41-ccf8-41e0-8e66-7dc9263edde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:58.867391 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 155/320] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 22 11:31:58.901144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129110ed-2146-4d42-9960-becdda33d199 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:58.986426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd4fbf-8ea6-4952-b4a7-61b8ca298586 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:58.987650 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd4fbf-8ea6-4952-b4a7-61b8ca298586 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:58.989131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd4fbf-8ea6-4952-b4a7-61b8ca298586 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:58.992703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 156/321] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:58 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:58.998449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a85d02-1e05-4748-b5ea-0a5eb61ae0dd None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:58.999100 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a85d02-1e05-4748-b5ea-0a5eb61ae0dd None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:58.999608 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a85d02-1e05-4748-b5ea-0a5eb61ae0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:59.024444 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f7a85d02-1e05-4748-b5ea-0a5eb61ae0dd None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:59.026149 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f7a85d02-1e05-4748-b5ea-0a5eb61ae0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:59.046016 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:59.131699 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 157/322] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:59.146153 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a85d02-1e05-4748-b5ea-0a5eb61ae0dd None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:59.149357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129110ed-2146-4d42-9960-becdda33d199 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:59.213590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986e62d-de72-4a11-acf1-7c50064f1106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:59.214739 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986e62d-de72-4a11-acf1-7c50064f1106 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:59.215189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986e62d-de72-4a11-acf1-7c50064f1106 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:59.215622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986e62d-de72-4a11-acf1-7c50064f1106 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:59.217202 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986e62d-de72-4a11-acf1-7c50064f1106 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:59.219428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986e62d-de72-4a11-acf1-7c50064f1106 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:59.223261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 166/323] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:31:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:59.228508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986e62d-de72-4a11-acf1-7c50064f1106 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:59.281844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a85d02-1e05-4748-b5ea-0a5eb61ae0dd None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:59.302629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:59.338302 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0697c3-4e23-43ca-a18b-e8187b1d6a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:59.340229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0697c3-4e23-43ca-a18b-e8187b1d6a25 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:59.340933 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0697c3-4e23-43ca-a18b-e8187b1d6a25 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:59.341561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0697c3-4e23-43ca-a18b-e8187b1d6a25 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:59.359267 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0697c3-4e23-43ca-a18b-e8187b1d6a25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:59.369304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-fa0697c3-4e23-43ca-a18b-e8187b1d6a25 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 22 11:31:59.369304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:59.369304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:59.369304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:59.369304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:59.369304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:59.369304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:59.369304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:59.369304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:59.369304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:59.369304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:59.369304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:59.369304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:59.369304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:59.369304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:59.369304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:59.370555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:59.370555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:59.370555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:59.370555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:59.370555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:59.370555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:59.370555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:59.370555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:59.370555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:59.370555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:59.370555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:59.370555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:59.370555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:59.370555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:59.370555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:59.370555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:59.372082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:59.372082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:59.372082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:59.372082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:59.372082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:59.372082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:59.372082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:59.372082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:59.372082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:59.372082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:59.372082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 22 11:31:59.372082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:31:59.376479 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 158/324] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:31:59.383683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0697c3-4e23-43ca-a18b-e8187b1d6a25 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:59.399026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986e62d-de72-4a11-acf1-7c50064f1106 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:59.403254 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:59.443245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11ef6f-f839-4c86-aec2-1ebb952dcc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:59.443967 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11ef6f-f839-4c86-aec2-1ebb952dcc71 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:59.444247 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11ef6f-f839-4c86-aec2-1ebb952dcc71 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:59.444521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11ef6f-f839-4c86-aec2-1ebb952dcc71 None admin] PATH_INFO: `/v3/groups` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:59.445230 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11ef6f-f839-4c86-aec2-1ebb952dcc71 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:59.446714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11ef6f-f839-4c86-aec2-1ebb952dcc71 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:59.469851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 167/325] 10.222.0.36 () {60 vars in 1134 bytes} [Wed Jun 22 11:31:59 2022] POST /identity/v3/groups => generated 226 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:31:59.471837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefda54-89ee-4479-bc74-bf0b22bef2e7 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:59.472460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefda54-89ee-4479-bc74-bf0b22bef2e7 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:59.472705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefda54-89ee-4479-bc74-bf0b22bef2e7 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:59.475305 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 168/326] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:31:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:31:59.483670 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc12a30-d26d-4f51-aafb-bac7a043e072 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:59.484155 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc12a30-d26d-4f51-aafb-bac7a043e072 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:59.484609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc12a30-d26d-4f51-aafb-bac7a043e072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:59.512102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-fbc12a30-d26d-4f51-aafb-bac7a043e072 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:31:59.514238 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-fbc12a30-d26d-4f51-aafb-bac7a043e072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:31:59.575366 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0697c3-4e23-43ca-a18b-e8187b1d6a25 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:59.621881 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 169/327] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:31:59.630642 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028303c6-00f7-4439-a1d2-30a2fbe68a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:59.631485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028303c6-00f7-4439-a1d2-30a2fbe68a56 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:59.631861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028303c6-00f7-4439-a1d2-30a2fbe68a56 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:59.632200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028303c6-00f7-4439-a1d2-30a2fbe68a56 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:59.633048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028303c6-00f7-4439-a1d2-30a2fbe68a56 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:31:59.633428 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028303c6-00f7-4439-a1d2-30a2fbe68a56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:59.635014 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028303c6-00f7-4439-a1d2-30a2fbe68a56 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:59.648734 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc12a30-d26d-4f51-aafb-bac7a043e072 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:59.650515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 159/328] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:31:59.657102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028303c6-00f7-4439-a1d2-30a2fbe68a56 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:59.693058 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:59.756368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:31:59.780011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc12a30-d26d-4f51-aafb-bac7a043e072 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:59.808312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028303c6-00f7-4439-a1d2-30a2fbe68a56 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:31:59.832123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6ca6c-02dd-4836-bcc8-84c4d60a8ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:59.833302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6ca6c-02dd-4836-bcc8-84c4d60a8ef7 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:59.833769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6ca6c-02dd-4836-bcc8-84c4d60a8ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:59.834252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6ca6c-02dd-4836-bcc8-84c4d60a8ef7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:59.852117 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad6ca6c-02dd-4836-bcc8-84c4d60a8ef7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:31:59.861504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-dad6ca6c-02dd-4836-bcc8-84c4d60a8ef7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:59.861504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:31:59.861504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:31:59.861504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:31:59.861504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:31:59.861504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:31:59.861504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:31:59.861504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:31:59.861504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:31:59.861504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:31:59.861504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:31:59.861504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:31:59.861504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:31:59.861504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:31:59.861504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:31:59.861504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:31:59.862444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:31:59.862444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:31:59.862444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:31:59.862444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:31:59.862444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:31:59.862444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:31:59.862444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:31:59.862444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:31:59.862444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:31:59.862444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:31:59.862444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:31:59.862444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:31:59.862444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:31:59.862444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:31:59.862444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:31:59.862444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:31:59.863215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:31:59.863215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:31:59.863215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:31:59.863215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:31:59.863215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:31:59.863215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:31:59.863215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:31:59.863215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:31:59.863215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:31:59.863215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:31:59.863215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:31:59.863215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:31:59.864537 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6375b-620b-4e0e-ab86-2f8d9f38d11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:31:59.864635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 170/329] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:31:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:31:59.864703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6375b-620b-4e0e-ab86-2f8d9f38d11a None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:31:59.864703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6375b-620b-4e0e-ab86-2f8d9f38d11a None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:31:59.864703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6375b-620b-4e0e-ab86-2f8d9f38d11a None admin] PATH_INFO: `/v3/users/8d351384dae546b5a750bba731b9cd09` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:31:59.865709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6375b-620b-4e0e-ab86-2f8d9f38d11a None admin] RBAC: Authorizing `identity:get_user(user_id=8d351384dae546b5a750bba731b9cd09)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:31:59.869229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6375b-620b-4e0e-ab86-2f8d9f38d11a None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:31:59.869950 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 160/330] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:31:59 2022] GET /identity/v3/users/8d351384dae546b5a750bba731b9cd09 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:31:59.875979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6ca6c-02dd-4836-bcc8-84c4d60a8ef7 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:31:59.878042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6375b-620b-4e0e-ab86-2f8d9f38d11a None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:00.041000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:00.053318 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6ca6c-02dd-4836-bcc8-84c4d60a8ef7 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:00.068135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6375b-620b-4e0e-ab86-2f8d9f38d11a None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:00.094978 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fd01e-5075-45f3-8314-d673a26fcc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:00.095967 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fd01e-5075-45f3-8314-d673a26fcc33 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:00.096739 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fd01e-5075-45f3-8314-d673a26fcc33 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:00.097170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fd01e-5075-45f3-8314-d673a26fcc33 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:00.098916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fd01e-5075-45f3-8314-d673a26fcc33 None admin] RBAC: Adding query filter params (name=member) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:00.099411 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fd01e-5075-45f3-8314-d673a26fcc33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:00.101360 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fd01e-5075-45f3-8314-d673a26fcc33 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:00.117636 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 171/331] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:31:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:00.120776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fd01e-5075-45f3-8314-d673a26fcc33 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:00.140414 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c4077-d463-4324-892a-8c9c1b2c1b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:00.142317 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c4077-d463-4324-892a-8c9c1b2c1b81 None admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:00.142823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c4077-d463-4324-892a-8c9c1b2c1b81 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:00.143318 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c4077-d463-4324-892a-8c9c1b2c1b81 None admin] PATH_INFO: `/v3/system/users/8d351384dae546b5a750bba731b9cd09/roles/2b63bb896abc4a22ba92b589f8ba2a73` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:00.146256 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c4077-d463-4324-892a-8c9c1b2c1b81 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8d351384dae546b5a750bba731b9cd09, role_id=2b63bb896abc4a22ba92b589f8ba2a73)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:00.149175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c4077-d463-4324-892a-8c9c1b2c1b81 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:00.157552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:00.170879 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 161/332] 10.222.0.36 () {58 vars in 1334 bytes} [Wed Jun 22 11:31:59 2022] PUT /identity/v3/system/users/8d351384dae546b5a750bba731b9cd09/roles/2b63bb896abc4a22ba92b589f8ba2a73 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:32:00.289715 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fd01e-5075-45f3-8314-d673a26fcc33 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:00.330820 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a53997-84e1-4dfb-945a-b8e734b360ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:00.331769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a53997-84e1-4dfb-945a-b8e734b360ba None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:00.332060 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a53997-84e1-4dfb-945a-b8e734b360ba None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:00.332394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a53997-84e1-4dfb-945a-b8e734b360ba None admin] PATH_INFO: `/v3/users/a58210d2064e416fb037849f65f8b617` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:00.334325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a53997-84e1-4dfb-945a-b8e734b360ba None admin] RBAC: Authorizing `identity:get_user(user_id=a58210d2064e416fb037849f65f8b617)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:00.335887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a53997-84e1-4dfb-945a-b8e734b360ba None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:00.338106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 172/333] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:32:00 2022] GET /identity/v3/users/a58210d2064e416fb037849f65f8b617 => generated 293 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:00.344298 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c4077-d463-4324-892a-8c9c1b2c1b81 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:00.527058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c4077-d463-4324-892a-8c9c1b2c1b81 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:00.570557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e5f54-21f3-4051-af75-f61da65c9c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:00.571242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e5f54-21f3-4051-af75-f61da65c9c71 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:00.571564 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e5f54-21f3-4051-af75-f61da65c9c71 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:00.571848 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e5f54-21f3-4051-af75-f61da65c9c71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:00.572715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e5f54-21f3-4051-af75-f61da65c9c71 None admin] RBAC: Adding query filter params (role.id=f2336919c5104f6a8210e99994f4d455, user.id=a58210d2064e416fb037849f65f8b617, scope.system=all) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:00.573160 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e5f54-21f3-4051-af75-f61da65c9c71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:00.574753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e5f54-21f3-4051-af75-f61da65c9c71 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:00.593683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 162/334] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Jun 22 11:32:00 2022] GET /identity/v3/role_assignments?user.id=a58210d2064e416fb037849f65f8b617&scope.system=all&role.id=f2336919c5104f6a8210e99994f4d455 => generated 510 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:02.197680 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e63ea-00a4-4dbb-a534-cbef4a24b3b2 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:02.198189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e63ea-00a4-4dbb-a534-cbef4a24b3b2 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:02.198189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e63ea-00a4-4dbb-a534-cbef4a24b3b2 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:02.199815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 173/335] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:32:02.205869 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e5524-f901-4199-b6f4-621713d3ac46 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:02.206374 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e5524-f901-4199-b6f4-621713d3ac46 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:02.206476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e5524-f901-4199-b6f4-621713d3ac46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:02.233320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-a10e5524-f901-4199-b6f4-621713d3ac46 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:02.234582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-a10e5524-f901-4199-b6f4-621713d3ac46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:02.309164 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:02.329268 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 163/336] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:02.368244 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e63ea-00a4-4dbb-a534-cbef4a24b3b2 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:02.403268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:02.524237 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e63ea-00a4-4dbb-a534-cbef4a24b3b2 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:02.581309 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820e98b-c879-41ac-bd6a-ffe855615f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:02.582169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820e98b-c879-41ac-bd6a-ffe855615f2b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:02.582451 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820e98b-c879-41ac-bd6a-ffe855615f2b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:02.583278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820e98b-c879-41ac-bd6a-ffe855615f2b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:02.602321 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820e98b-c879-41ac-bd6a-ffe855615f2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:02.611642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-8820e98b-c879-41ac-bd6a-ffe855615f2b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:32:02.611642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:02.611642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:02.611642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:02.611642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:02.611642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:02.611642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:02.611642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:02.611642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:02.611642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:02.611642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:02.611642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:02.611642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:02.611642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:02.611642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:02.611642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:02.612888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:02.612888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:02.612888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:02.612888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:02.612888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:02.612888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:02.612888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:02.612888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:02.612888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:02.612888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:02.612888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:02.612888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:02.612888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:02.612888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:02.612888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:02.612888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:02.614719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:02.614719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:02.614719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:02.614719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:02.614719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:02.614719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:02.614719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:02.614719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:02.614719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:02.614719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:02.614719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:32:02.614719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:02.616308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 174/337] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:32:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:02.620777 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e5524-f901-4199-b6f4-621713d3ac46 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:02.822929 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e5524-f901-4199-b6f4-621713d3ac46 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:02.899729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bd7e9-14e8-483e-866a-5e3919cb78b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:02.901043 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bd7e9-14e8-483e-866a-5e3919cb78b7 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:02.901729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bd7e9-14e8-483e-866a-5e3919cb78b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:02.902322 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bd7e9-14e8-483e-866a-5e3919cb78b7 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:02.903367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501bd7e9-14e8-483e-866a-5e3919cb78b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:02.904010 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501bd7e9-14e8-483e-866a-5e3919cb78b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:02.906352 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501bd7e9-14e8-483e-866a-5e3919cb78b7 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:02.924325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 164/338] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:02.943700 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820e98b-c879-41ac-bd6a-ffe855615f2b None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:03.069577 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:03.099406 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820e98b-c879-41ac-bd6a-ffe855615f2b None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:03.138623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aba4da-149e-42de-af38-3cc6dd35e032 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:03.139492 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aba4da-149e-42de-af38-3cc6dd35e032 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:03.139492 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aba4da-149e-42de-af38-3cc6dd35e032 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:03.141645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 165/339] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:03.146186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bac1f-292a-40ef-be00-b88709051a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:03.146186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bac1f-292a-40ef-be00-b88709051a1d None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:03.146186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bac1f-292a-40ef-be00-b88709051a1d None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:03.146186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bac1f-292a-40ef-be00-b88709051a1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:03.149198 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e55c1b-b704-45c9-bf90-1ace0382f8e8 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:03.150106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e55c1b-b704-45c9-bf90-1ace0382f8e8 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:03.150868 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e55c1b-b704-45c9-bf90-1ace0382f8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:03.208867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-31e55c1b-b704-45c9-bf90-1ace0382f8e8 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:03.212490 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-31e55c1b-b704-45c9-bf90-1ace0382f8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:03.237412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1bac1f-292a-40ef-be00-b88709051a1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:03.240289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1bac1f-292a-40ef-be00-b88709051a1d None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:03.335975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:03.381224 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:03.390903 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 166/340] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:03.423938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e55c1b-b704-45c9-bf90-1ace0382f8e8 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:03.425396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 175/341] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:03.438673 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bac1f-292a-40ef-be00-b88709051a1d None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:03.629917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bac1f-292a-40ef-be00-b88709051a1d None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:03.655760 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e55c1b-b704-45c9-bf90-1ace0382f8e8 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:03.697918 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecad3c-5db1-4d7c-be6c-2795fa881493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:03.698967 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecad3c-5db1-4d7c-be6c-2795fa881493 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:03.699251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecad3c-5db1-4d7c-be6c-2795fa881493 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:03.699551 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecad3c-5db1-4d7c-be6c-2795fa881493 None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:03.701590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecad3c-5db1-4d7c-be6c-2795fa881493 None admin] RBAC: Authorizing `identity:get_project(project_id=e263f641bede4564a5855da9ec8514ed)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:03.703853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecad3c-5db1-4d7c-be6c-2795fa881493 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:03.706344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 176/342] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:32:03 2022] GET /identity/v3/projects/e263f641bede4564a5855da9ec8514ed => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:03.715207 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecad3c-5db1-4d7c-be6c-2795fa881493 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:03.724278 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c9b9b-190d-4a43-aa33-ebdf6b8f7b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:03.725487 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c9b9b-190d-4a43-aa33-ebdf6b8f7b11 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:03.725975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c9b9b-190d-4a43-aa33-ebdf6b8f7b11 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:03.726452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c9b9b-190d-4a43-aa33-ebdf6b8f7b11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:03.752484 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c9b9b-190d-4a43-aa33-ebdf6b8f7b11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:03.761747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-364c9b9b-190d-4a43-aa33-ebdf6b8f7b11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:32:03.761747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:03.761747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:03.761747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:03.761747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:03.761747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:03.761747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:03.761747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:03.761747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:03.761747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:03.761747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:03.761747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:03.761747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:03.761747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:03.761747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:03.761747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:03.762719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:03.762719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:03.762719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:03.762719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:03.762719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:03.762719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:03.762719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:03.762719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:03.762719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:03.762719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:03.762719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:03.762719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:03.762719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:03.762719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:03.762719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:03.762719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:03.763561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:03.763561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:03.763561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:03.763561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:03.763561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:03.763561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:03.763561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:03.763561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:03.763561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:03.763561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:03.763561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:32:03.763561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:03.771880 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 167/343] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:03.777414 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c9b9b-190d-4a43-aa33-ebdf6b8f7b11 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:03.811863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:03.846273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:03.947039 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecad3c-5db1-4d7c-be6c-2795fa881493 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:03.985402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c9b9b-190d-4a43-aa33-ebdf6b8f7b11 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:04.015522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824f40f-b459-47cb-8963-3bfe175b2ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:04.016927 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824f40f-b459-47cb-8963-3bfe175b2ba3 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:04.017426 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824f40f-b459-47cb-8963-3bfe175b2ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:04.017905 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824f40f-b459-47cb-8963-3bfe175b2ba3 None admin] PATH_INFO: `/v3/groups/b58cdca6be124c00b0deece5538c211c` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:04.030011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824f40f-b459-47cb-8963-3bfe175b2ba3 None admin] RBAC: Authorizing `identity:get_group(group_id=b58cdca6be124c00b0deece5538c211c)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:04.034954 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824f40f-b459-47cb-8963-3bfe175b2ba3 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:04.042337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 177/344] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jun 22 11:32:03 2022] GET /identity/v3/groups/b58cdca6be124c00b0deece5538c211c => generated 226 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:04.044306 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824f40f-b459-47cb-8963-3bfe175b2ba3 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:04.116573 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d042ef6-01a0-4772-b8ba-7db2c2c6d9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:04.118003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d042ef6-01a0-4772-b8ba-7db2c2c6d9aa None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:04.118658 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d042ef6-01a0-4772-b8ba-7db2c2c6d9aa None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:04.119177 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d042ef6-01a0-4772-b8ba-7db2c2c6d9aa None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:04.120319 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d042ef6-01a0-4772-b8ba-7db2c2c6d9aa None admin] RBAC: Adding query filter params (name=member) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:04.120811 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d042ef6-01a0-4772-b8ba-7db2c2c6d9aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:04.123023 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d042ef6-01a0-4772-b8ba-7db2c2c6d9aa None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:04.150049 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 168/345] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:32:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:04.152754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75cb76e-a778-4353-9c84-37d717f34bec None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:04.153555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75cb76e-a778-4353-9c84-37d717f34bec None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:04.154056 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75cb76e-a778-4353-9c84-37d717f34bec None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:04.161468 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 169/346] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:04.162390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75cb76e-a778-4353-9c84-37d717f34bec None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:04.251483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824f40f-b459-47cb-8963-3bfe175b2ba3 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:04.283204 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:04.314778 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75cb76e-a778-4353-9c84-37d717f34bec None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:04.321148 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab898ba-46ea-4818-aee8-46fe1aaec8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:04.322214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab898ba-46ea-4818-aee8-46fe1aaec8f7 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:04.322616 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab898ba-46ea-4818-aee8-46fe1aaec8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:04.322997 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab898ba-46ea-4818-aee8-46fe1aaec8f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:04.325533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab898ba-46ea-4818-aee8-46fe1aaec8f7 None admin] RBAC: Adding query filter params (group.id=b58cdca6be124c00b0deece5538c211c, role.id=b4be7021e4104740b8b0b3b40ecd3d59, scope.project.id=e263f641bede4564a5855da9ec8514ed) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:04.326265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab898ba-46ea-4818-aee8-46fe1aaec8f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:04.328234 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab898ba-46ea-4818-aee8-46fe1aaec8f7 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:04.343623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 178/347] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jun 22 11:32:04 2022] GET /identity/v3/role_assignments?group.id=b58cdca6be124c00b0deece5538c211c&scope.project.id=e263f641bede4564a5855da9ec8514ed&role.id=b4be7021e4104740b8b0b3b40ecd3d59 => generated 260 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:04.343823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:04.346003 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e477b43-f9e6-43d7-8c81-f6e9f5705455 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:04.346796 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e477b43-f9e6-43d7-8c81-f6e9f5705455 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:04.347843 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e477b43-f9e6-43d7-8c81-f6e9f5705455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:04.367497 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d8496-6ada-494c-ba0f-c67262cd8ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:04.370426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d8496-6ada-494c-ba0f-c67262cd8ab9 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:04.370736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d8496-6ada-494c-ba0f-c67262cd8ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:04.370934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d8496-6ada-494c-ba0f-c67262cd8ab9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:04.389785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-5e477b43-f9e6-43d7-8c81-f6e9f5705455 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:04.394315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-5e477b43-f9e6-43d7-8c81-f6e9f5705455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:04.487268 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d8496-6ada-494c-ba0f-c67262cd8ab9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:04.489614 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d8496-6ada-494c-ba0f-c67262cd8ab9 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:04.597812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 179/348] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:04.598973 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1a66c-32cb-4ed5-93f5-95af59d70f94 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:04.599262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1a66c-32cb-4ed5-93f5-95af59d70f94 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:04.599566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1a66c-32cb-4ed5-93f5-95af59d70f94 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:04.600951 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 180/349] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:04.616893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cf5eb-8774-40f6-aba7-84ccc7a08c2a None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:04.617440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cf5eb-8774-40f6-aba7-84ccc7a08c2a None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:04.617862 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cf5eb-8774-40f6-aba7-84ccc7a08c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:04.631403 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 170/350] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:04.632558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d8496-6ada-494c-ba0f-c67262cd8ab9 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:04.649114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:04.651490 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-881cf5eb-8774-40f6-aba7-84ccc7a08c2a None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:04.652711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-881cf5eb-8774-40f6-aba7-84ccc7a08c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:04.733370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 181/351] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:04.734368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881cf5eb-8774-40f6-aba7-84ccc7a08c2a None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:04.759612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d8496-6ada-494c-ba0f-c67262cd8ab9 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:04.799676 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:04.810733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd51cd-46a7-4d1f-a6c6-9038bed6afd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:04.812633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd51cd-46a7-4d1f-a6c6-9038bed6afd9 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:04.813408 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd51cd-46a7-4d1f-a6c6-9038bed6afd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:04.814147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd51cd-46a7-4d1f-a6c6-9038bed6afd9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:04.838087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd51cd-46a7-4d1f-a6c6-9038bed6afd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:04.854224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-81bd51cd-46a7-4d1f-a6c6-9038bed6afd9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 22 11:32:04.854224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:04.854224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:04.854224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:04.854224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:04.854224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:04.854224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:04.854224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:04.854224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:04.854224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:04.854224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:04.854224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:04.854224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:04.854224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:04.854224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:04.854224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:04.856038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:04.856038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:04.856038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:04.856038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:04.856038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:04.856038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:04.856038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:04.856038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:04.856038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:04.856038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:04.856038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:04.856038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:04.856038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:04.856038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:04.856038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:04.856038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:04.857621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:04.857621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:04.857621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:04.857621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:04.857621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:04.857621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:04.857621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:04.857621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:04.857621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:04.857621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:04.857621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 22 11:32:04.857621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:04.861915 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 171/352] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:04.865123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d777aaa-0e29-4a4d-a7be-01850e11363a None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:04.865963 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d777aaa-0e29-4a4d-a7be-01850e11363a None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:04.866683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d777aaa-0e29-4a4d-a7be-01850e11363a None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:04.869495 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 172/353] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:04.871204 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d777aaa-0e29-4a4d-a7be-01850e11363a None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:04.936101 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881cf5eb-8774-40f6-aba7-84ccc7a08c2a None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:04.998394 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:05.018003 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310361ff-c936-4fa4-be01-d3c0e3db3c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:05.018893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310361ff-c936-4fa4-be01-d3c0e3db3c76 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:05.019181 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310361ff-c936-4fa4-be01-d3c0e3db3c76 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:05.019477 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310361ff-c936-4fa4-be01-d3c0e3db3c76 None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:05.022528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310361ff-c936-4fa4-be01-d3c0e3db3c76 None admin] RBAC: Authorizing `identity:get_project(project_id=e263f641bede4564a5855da9ec8514ed)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:05.024056 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310361ff-c936-4fa4-be01-d3c0e3db3c76 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:05.026165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 182/354] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:32:04 2022] GET /identity/v3/projects/e263f641bede4564a5855da9ec8514ed => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:05.026782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310361ff-c936-4fa4-be01-d3c0e3db3c76 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:05.080769 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d777aaa-0e29-4a4d-a7be-01850e11363a None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:05.138452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995de41-0fb3-41f9-bb91-a31a1b18ddc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:05.139203 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995de41-0fb3-41f9-bb91-a31a1b18ddc3 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:05.141958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995de41-0fb3-41f9-bb91-a31a1b18ddc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:05.142322 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995de41-0fb3-41f9-bb91-a31a1b18ddc3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:05.163258 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995de41-0fb3-41f9-bb91-a31a1b18ddc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:05.172062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-7995de41-0fb3-41f9-bb91-a31a1b18ddc3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:32:05.172062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:05.172062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:05.172062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:05.172062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:05.172062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:05.172062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:05.172062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:05.172062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:05.172062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:05.172062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:05.172062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:05.172062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:05.172062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:05.172062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:05.172062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:05.173928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:05.173928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:05.173928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:05.173928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:05.173928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:05.173928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:05.173928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:05.173928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:05.173928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:05.173928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:05.173928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:05.173928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:05.173928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:05.173928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:05.173928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:05.173928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:05.175613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:05.175613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:05.175613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:05.175613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:05.175613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:05.175613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:05.175613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:05.175613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:05.175613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:05.175613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:05.175613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:32:05.175613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:05.175613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 173/355] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:05.175613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5376d-faa7-40bd-8c99-2ef6004ba9c1 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:05.178733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5376d-faa7-40bd-8c99-2ef6004ba9c1 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:05.178733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5376d-faa7-40bd-8c99-2ef6004ba9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:05.196804 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-3be5376d-faa7-40bd-8c99-2ef6004ba9c1 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:05.198193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-3be5376d-faa7-40bd-8c99-2ef6004ba9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:05.206917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310361ff-c936-4fa4-be01-d3c0e3db3c76 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:05.224166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:05.291538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e71b6-4df6-4ac9-aa59-4cdff7a33413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:05.292863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e71b6-4df6-4ac9-aa59-4cdff7a33413 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:05.293483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e71b6-4df6-4ac9-aa59-4cdff7a33413 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:05.294146 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e71b6-4df6-4ac9-aa59-4cdff7a33413 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:05.295978 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e71b6-4df6-4ac9-aa59-4cdff7a33413 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:05.296742 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e71b6-4df6-4ac9-aa59-4cdff7a33413 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:05.299701 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e71b6-4df6-4ac9-aa59-4cdff7a33413 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:05.309901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 174/356] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:05.312654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8776c-b276-49af-826c-d2dce816fbba None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:05.313523 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8776c-b276-49af-826c-d2dce816fbba None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:05.314293 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8776c-b276-49af-826c-d2dce816fbba None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:05.317819 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 175/357] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:05.320403 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8776c-b276-49af-826c-d2dce816fbba None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:05.321581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 183/358] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:32:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:05.327510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e71b6-4df6-4ac9-aa59-4cdff7a33413 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:05.439112 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:05.518419 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8776c-b276-49af-826c-d2dce816fbba None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:05.527332 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e71b6-4df6-4ac9-aa59-4cdff7a33413 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:05.581060 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaede3d7-c25f-4991-9bff-74ce1f92d030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:05.581959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaede3d7-c25f-4991-9bff-74ce1f92d030 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:05.582260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaede3d7-c25f-4991-9bff-74ce1f92d030 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:05.583034 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaede3d7-c25f-4991-9bff-74ce1f92d030 None admin] PATH_INFO: `/v3/groups/b58cdca6be124c00b0deece5538c211c` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:05.584454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaede3d7-c25f-4991-9bff-74ce1f92d030 None admin] RBAC: Authorizing `identity:get_group(group_id=b58cdca6be124c00b0deece5538c211c)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:05.586082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaede3d7-c25f-4991-9bff-74ce1f92d030 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:05.588244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 176/359] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jun 22 11:32:05 2022] GET /identity/v3/groups/b58cdca6be124c00b0deece5538c211c => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:05.589279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd3598-5928-434f-a579-e077cc5a6289 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:05.589571 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd3598-5928-434f-a579-e077cc5a6289 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:05.589860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd3598-5928-434f-a579-e077cc5a6289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:05.600028 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd20f6-a2fc-44f7-9349-65b53f9d3f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:05.601735 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd20f6-a2fc-44f7-9349-65b53f9d3f5b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:05.602272 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd20f6-a2fc-44f7-9349-65b53f9d3f5b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:05.602795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd20f6-a2fc-44f7-9349-65b53f9d3f5b None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:05.604321 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd20f6-a2fc-44f7-9349-65b53f9d3f5b None admin] RBAC: Adding query filter params (name=member) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:05.604891 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd20f6-a2fc-44f7-9349-65b53f9d3f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:05.606813 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd20f6-a2fc-44f7-9349-65b53f9d3f5b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:05.608480 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-cddd3598-5928-434f-a579-e077cc5a6289 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:05.609827 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-cddd3598-5928-434f-a579-e077cc5a6289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:05.618289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 184/360] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:32:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:05.619526 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd20f6-a2fc-44f7-9349-65b53f9d3f5b None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:05.665757 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:05.735217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 177/361] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:05.736139 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd3598-5928-434f-a579-e077cc5a6289 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:05.776658 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd20f6-a2fc-44f7-9349-65b53f9d3f5b None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:05.833196 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac3455-b3f7-43ca-8124-0c11fc8ce4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:05.834096 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac3455-b3f7-43ca-8124-0c11fc8ce4de None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:05.834382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac3455-b3f7-43ca-8124-0c11fc8ce4de None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:05.834663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac3455-b3f7-43ca-8124-0c11fc8ce4de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:05.846870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:05.855643 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac3455-b3f7-43ca-8124-0c11fc8ce4de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:05.865839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-e1ac3455-b3f7-43ca-8124-0c11fc8ce4de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:32:05.865839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:05.865839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:05.865839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:05.865839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:05.865839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:05.865839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:05.865839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:05.865839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:05.865839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:05.865839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:05.865839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:05.865839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:05.865839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:05.865839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:05.865839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:05.867606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:05.867606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:05.867606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:05.867606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:05.867606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:05.867606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:05.867606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:05.867606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:05.867606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:05.867606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:05.867606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:05.867606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:05.867606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:05.867606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:05.867606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:05.867606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:05.868721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:05.868721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:05.868721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:05.868721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:05.868721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:05.868721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:05.868721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:05.868721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:05.868721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:05.868721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:05.868721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:32:05.868721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:05.868721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 185/362] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:32:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:05.868721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac3455-b3f7-43ca-8124-0c11fc8ce4de None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:05.925909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd3598-5928-434f-a579-e077cc5a6289 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:05.973880 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb86714c-c8b7-436e-b10f-84f5470c2b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:05.975075 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86714c-c8b7-436e-b10f-84f5470c2b5d None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:05.975568 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86714c-c8b7-436e-b10f-84f5470c2b5d None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:05.975989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86714c-c8b7-436e-b10f-84f5470c2b5d None admin] PATH_INFO: `/v3/users/8d351384dae546b5a750bba731b9cd09` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:05.977528 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb86714c-c8b7-436e-b10f-84f5470c2b5d None admin] RBAC: Authorizing `identity:get_user(user_id=8d351384dae546b5a750bba731b9cd09)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:05.979066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb86714c-c8b7-436e-b10f-84f5470c2b5d None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:05.981120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 178/363] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:32:05 2022] GET /identity/v3/users/8d351384dae546b5a750bba731b9cd09 => generated 293 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:05.981539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb86714c-c8b7-436e-b10f-84f5470c2b5d None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:06.024067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:06.040517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac3455-b3f7-43ca-8124-0c11fc8ce4de None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:06.084002 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e8525-59b6-49e9-a98a-80e927556ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:06.084856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e8525-59b6-49e9-a98a-80e927556ad5 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:06.085290 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e8525-59b6-49e9-a98a-80e927556ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:06.085638 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e8525-59b6-49e9-a98a-80e927556ad5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:06.086670 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e8525-59b6-49e9-a98a-80e927556ad5 None admin] RBAC: Adding query filter params (group.id=b58cdca6be124c00b0deece5538c211c, role.id=f2336919c5104f6a8210e99994f4d455, scope.project.id=e263f641bede4564a5855da9ec8514ed) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:06.087053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e8525-59b6-49e9-a98a-80e927556ad5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:06.088631 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e8525-59b6-49e9-a98a-80e927556ad5 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:06.101587 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 186/364] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jun 22 11:32:05 2022] GET /identity/v3/role_assignments?group.id=b58cdca6be124c00b0deece5538c211c&scope.project.id=e263f641bede4564a5855da9ec8514ed&role.id=f2336919c5104f6a8210e99994f4d455 => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:06.103217 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e8525-59b6-49e9-a98a-80e927556ad5 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:06.186309 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb86714c-c8b7-436e-b10f-84f5470c2b5d None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:06.211624 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:06.241537 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d197830-45fd-45ba-8567-ca8d7f2a000e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:06.242347 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d197830-45fd-45ba-8567-ca8d7f2a000e None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:06.242646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d197830-45fd-45ba-8567-ca8d7f2a000e None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:06.242936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d197830-45fd-45ba-8567-ca8d7f2a000e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:06.266798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e8525-59b6-49e9-a98a-80e927556ad5 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:06.308579 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b78af-0c98-47c9-8d87-8202b470ca24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:06.309313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b78af-0c98-47c9-8d87-8202b470ca24 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:06.309587 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b78af-0c98-47c9-8d87-8202b470ca24 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:06.309865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b78af-0c98-47c9-8d87-8202b470ca24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:06.322202 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b78af-0c98-47c9-8d87-8202b470ca24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:06.327842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d197830-45fd-45ba-8567-ca8d7f2a000e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:06.329388 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d197830-45fd-45ba-8567-ca8d7f2a000e None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:06.331419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-6a5b78af-0c98-47c9-8d87-8202b470ca24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:32:06.331419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:06.331419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:06.331419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:06.331419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:06.331419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:06.331419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:06.331419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:06.331419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:06.331419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:06.331419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:06.331419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:06.331419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:06.331419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:06.331419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:06.331419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:06.332325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:06.332325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:06.332325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:06.332325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:06.332325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:06.332325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:06.332325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:06.332325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:06.332325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:06.332325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:06.332325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:06.332325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:06.332325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:06.332325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:06.332325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:06.332325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:06.333656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:06.333656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:06.333656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:06.333656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:06.333656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:06.333656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:06.333656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:06.333656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:06.333656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:06.333656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:06.333656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:32:06.333656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:06.333656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 187/365] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:32:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:06.334329 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b78af-0c98-47c9-8d87-8202b470ca24 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:06.361534 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:06.484693 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 179/366] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:06.485578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d197830-45fd-45ba-8567-ca8d7f2a000e None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:06.511176 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:06.536606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b78af-0c98-47c9-8d87-8202b470ca24 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:06.600111 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e9f03-f53a-46ed-bd4f-0f6cc1b05428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:06.601222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e9f03-f53a-46ed-bd4f-0f6cc1b05428 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:06.601726 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e9f03-f53a-46ed-bd4f-0f6cc1b05428 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:06.602242 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e9f03-f53a-46ed-bd4f-0f6cc1b05428 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:06.603238 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e9f03-f53a-46ed-bd4f-0f6cc1b05428 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:06.603803 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e9f03-f53a-46ed-bd4f-0f6cc1b05428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:06.605465 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e9f03-f53a-46ed-bd4f-0f6cc1b05428 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:06.618270 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 188/367] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:06.619577 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e9f03-f53a-46ed-bd4f-0f6cc1b05428 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:06.644141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d197830-45fd-45ba-8567-ca8d7f2a000e None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:06.683854 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d28526-a631-413f-80cc-11c9f91c065c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:06.684577 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d28526-a631-413f-80cc-11c9f91c065c None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:06.684869 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d28526-a631-413f-80cc-11c9f91c065c None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:06.685142 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d28526-a631-413f-80cc-11c9f91c065c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:06.686000 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d28526-a631-413f-80cc-11c9f91c065c None admin] RBAC: Adding query filter params (role.id=2b63bb896abc4a22ba92b589f8ba2a73, user.id=8d351384dae546b5a750bba731b9cd09, scope.system=all) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:06.686312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d28526-a631-413f-80cc-11c9f91c065c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:06.687940 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d28526-a631-413f-80cc-11c9f91c065c None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:06.704406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 180/368] 10.222.0.36 () {56 vars in 1308 bytes} [Wed Jun 22 11:32:06 2022] GET /identity/v3/role_assignments?user.id=8d351384dae546b5a750bba731b9cd09&scope.system=all&role.id=2b63bb896abc4a22ba92b589f8ba2a73 => generated 510 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:06.705088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d28526-a631-413f-80cc-11c9f91c065c None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:06.719509 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:06.758312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e9f03-f53a-46ed-bd4f-0f6cc1b05428 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:06.803206 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab66fbe-ff13-4459-840d-059e9bdd0180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:06.804322 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab66fbe-ff13-4459-840d-059e9bdd0180 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:06.804815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab66fbe-ff13-4459-840d-059e9bdd0180 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:06.805331 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab66fbe-ff13-4459-840d-059e9bdd0180 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:06.806693 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab66fbe-ff13-4459-840d-059e9bdd0180 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:06.807243 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab66fbe-ff13-4459-840d-059e9bdd0180 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:06.808926 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab66fbe-ff13-4459-840d-059e9bdd0180 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:06.814782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:06.820003 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 189/369] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:06.821243 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab66fbe-ff13-4459-840d-059e9bdd0180 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:06.845064 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d28526-a631-413f-80cc-11c9f91c065c None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:06.890977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fed89-1a57-4454-b707-492844cc70ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:06.892408 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fed89-1a57-4454-b707-492844cc70ce None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:06.892880 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fed89-1a57-4454-b707-492844cc70ce None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:06.893519 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fed89-1a57-4454-b707-492844cc70ce None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:06.895921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fed89-1a57-4454-b707-492844cc70ce None admin] RBAC: Authorizing `identity:get_project(project_id=a89544d6d9d14e759c077d7175365dca)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:06.899082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fed89-1a57-4454-b707-492844cc70ce None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:06.902645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 181/370] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:32:06 2022] GET /identity/v3/projects/a89544d6d9d14e759c077d7175365dca => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:06.904290 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fed89-1a57-4454-b707-492844cc70ce None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:06.954772 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab66fbe-ff13-4459-840d-059e9bdd0180 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:06.993992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f07e3-4b87-4c4a-9645-e2c112edeaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:06.994782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f07e3-4b87-4c4a-9645-e2c112edeaed None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:06.995133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f07e3-4b87-4c4a-9645-e2c112edeaed None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:06.995512 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f07e3-4b87-4c4a-9645-e2c112edeaed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:07.007546 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:07.041212 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fed89-1a57-4454-b707-492844cc70ce None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:07.077002 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f07e3-4b87-4c4a-9645-e2c112edeaed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:07.078510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f07e3-4b87-4c4a-9645-e2c112edeaed None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:07.087063 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24441f9-6c98-4aa1-b5ba-d81afaddc655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:07.088038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24441f9-6c98-4aa1-b5ba-d81afaddc655 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:07.088475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24441f9-6c98-4aa1-b5ba-d81afaddc655 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:07.089047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24441f9-6c98-4aa1-b5ba-d81afaddc655 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:07.107432 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:07.182750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24441f9-6c98-4aa1-b5ba-d81afaddc655 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:07.185005 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24441f9-6c98-4aa1-b5ba-d81afaddc655 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:07.219539 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 190/371] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:07.220860 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f07e3-4b87-4c4a-9645-e2c112edeaed None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:07.268040 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:07.327131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 182/372] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:07.328630 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43ebd5-05f6-46f7-b7ba-3a57ba232053 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:07.329160 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43ebd5-05f6-46f7-b7ba-3a57ba232053 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:07.329301 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43ebd5-05f6-46f7-b7ba-3a57ba232053 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:07.331201 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 183/373] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:07.332329 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43ebd5-05f6-46f7-b7ba-3a57ba232053 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:07.344803 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f07e3-4b87-4c4a-9645-e2c112edeaed None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:07.385219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652082cb-02e9-4383-9ac2-7bbc7f07a0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:07.386084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652082cb-02e9-4383-9ac2-7bbc7f07a0ab None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:07.386419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652082cb-02e9-4383-9ac2-7bbc7f07a0ab None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:07.386704 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652082cb-02e9-4383-9ac2-7bbc7f07a0ab None admin] PATH_INFO: `/v3/groups/b58cdca6be124c00b0deece5538c211c` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:07.388751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652082cb-02e9-4383-9ac2-7bbc7f07a0ab None admin] RBAC: Authorizing `identity:get_group(group_id=b58cdca6be124c00b0deece5538c211c)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:07.389463 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652082cb-02e9-4383-9ac2-7bbc7f07a0ab None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:07.391419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 191/374] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jun 22 11:32:07 2022] GET /identity/v3/groups/b58cdca6be124c00b0deece5538c211c => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:07.396259 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652082cb-02e9-4383-9ac2-7bbc7f07a0ab None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:07.403773 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:07.469914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43ebd5-05f6-46f7-b7ba-3a57ba232053 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:07.513498 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf1146-9302-40f8-bd08-b9e6b42c7294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:07.514380 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf1146-9302-40f8-bd08-b9e6b42c7294 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:07.514663 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf1146-9302-40f8-bd08-b9e6b42c7294 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:07.514909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf1146-9302-40f8-bd08-b9e6b42c7294 None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:07.516494 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf1146-9302-40f8-bd08-b9e6b42c7294 None admin] RBAC: Authorizing `identity:get_project(project_id=a89544d6d9d14e759c077d7175365dca)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:07.517989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf1146-9302-40f8-bd08-b9e6b42c7294 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:07.520949 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 184/375] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:32:07 2022] GET /identity/v3/projects/a89544d6d9d14e759c077d7175365dca => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:07.522014 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5797e-4965-4954-9853-3b8629749efd None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:07.522329 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5797e-4965-4954-9853-3b8629749efd None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:07.522581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5797e-4965-4954-9853-3b8629749efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:07.530195 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652082cb-02e9-4383-9ac2-7bbc7f07a0ab None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:07.538258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:07.545285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f7a5797e-4965-4954-9853-3b8629749efd None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:07.546922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f7a5797e-4965-4954-9853-3b8629749efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:07.569349 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c1925-89d6-4a82-9d3c-7e81cfa24d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:07.570352 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c1925-89d6-4a82-9d3c-7e81cfa24d9e None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:07.570634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c1925-89d6-4a82-9d3c-7e81cfa24d9e None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:07.570910 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c1925-89d6-4a82-9d3c-7e81cfa24d9e None admin] PATH_INFO: `/v3/projects/c77776cd83344a69a0a4fbb7efadd00f` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:07.572467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c1925-89d6-4a82-9d3c-7e81cfa24d9e None admin] RBAC: Authorizing `identity:get_project(project_id=c77776cd83344a69a0a4fbb7efadd00f)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:07.573915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c1925-89d6-4a82-9d3c-7e81cfa24d9e None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:07.576438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 192/376] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:32:07 2022] GET /identity/v3/projects/c77776cd83344a69a0a4fbb7efadd00f => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:07.578714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c1925-89d6-4a82-9d3c-7e81cfa24d9e None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:07.642608 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 185/377] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:07.643492 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5797e-4965-4954-9853-3b8629749efd None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:07.702532 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:07.738096 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c1925-89d6-4a82-9d3c-7e81cfa24d9e None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:07.779783 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5797e-4965-4954-9853-3b8629749efd None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:07.784794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df45ed2-ed98-4dd9-ae9e-7212513b4ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:07.785463 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df45ed2-ed98-4dd9-ae9e-7212513b4ea0 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:07.785748 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df45ed2-ed98-4dd9-ae9e-7212513b4ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:07.786057 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df45ed2-ed98-4dd9-ae9e-7212513b4ea0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:07.786888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df45ed2-ed98-4dd9-ae9e-7212513b4ea0 None admin] RBAC: Adding query filter params (group.id=b58cdca6be124c00b0deece5538c211c, role.id=f2336919c5104f6a8210e99994f4d455, scope.project.id=a89544d6d9d14e759c077d7175365dca) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:07.787201 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df45ed2-ed98-4dd9-ae9e-7212513b4ea0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:07.789203 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df45ed2-ed98-4dd9-ae9e-7212513b4ea0 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:07.795521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 193/378] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jun 22 11:32:07 2022] GET /identity/v3/role_assignments?group.id=b58cdca6be124c00b0deece5538c211c&scope.project.id=a89544d6d9d14e759c077d7175365dca&role.id=f2336919c5104f6a8210e99994f4d455 => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:07.796297 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df45ed2-ed98-4dd9-ae9e-7212513b4ea0 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:07.826957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ec1f0-fd50-42e0-831b-bce6c8493100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:07.828186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ec1f0-fd50-42e0-831b-bce6c8493100 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:07.828506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ec1f0-fd50-42e0-831b-bce6c8493100 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:07.829329 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ec1f0-fd50-42e0-831b-bce6c8493100 None admin] PATH_INFO: `/v3/groups/b58cdca6be124c00b0deece5538c211c` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:07.830993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ec1f0-fd50-42e0-831b-bce6c8493100 None admin] RBAC: Authorizing `identity:get_group(group_id=b58cdca6be124c00b0deece5538c211c)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:07.833564 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ec1f0-fd50-42e0-831b-bce6c8493100 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:07.836578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 186/379] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jun 22 11:32:07 2022] GET /identity/v3/groups/b58cdca6be124c00b0deece5538c211c => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:07.837441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ec1f0-fd50-42e0-831b-bce6c8493100 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:07.853639 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:07.956416 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df45ed2-ed98-4dd9-ae9e-7212513b4ea0 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:07.987785 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ec1f0-fd50-42e0-831b-bce6c8493100 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:08.003837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:08.006675 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbb036-3a0e-452b-97e6-69b8c55990f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:08.007650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbb036-3a0e-452b-97e6-69b8c55990f5 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:08.008154 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbb036-3a0e-452b-97e6-69b8c55990f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:08.008512 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbb036-3a0e-452b-97e6-69b8c55990f5 None admin] PATH_INFO: `/v3/groups/8521ee997e8847b6bbeb83507f355e93` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:08.015294 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbb036-3a0e-452b-97e6-69b8c55990f5 None admin] RBAC: Authorizing `identity:get_group(group_id=8521ee997e8847b6bbeb83507f355e93)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:08.017093 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bbb036-3a0e-452b-97e6-69b8c55990f5 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:08.019343 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 194/380] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jun 22 11:32:07 2022] GET /identity/v3/groups/8521ee997e8847b6bbeb83507f355e93 => generated 229 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:08.020733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbb036-3a0e-452b-97e6-69b8c55990f5 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:08.036755 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b5096-9088-4860-82a5-00966ead3ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:08.037943 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b5096-9088-4860-82a5-00966ead3ba5 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:08.038435 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b5096-9088-4860-82a5-00966ead3ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:08.038941 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b5096-9088-4860-82a5-00966ead3ba5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:08.053505 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b5096-9088-4860-82a5-00966ead3ba5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:08.065320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-856b5096-9088-4860-82a5-00966ead3ba5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:32:08.065320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:08.065320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:08.065320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:08.065320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:08.065320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:08.065320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:08.065320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:08.065320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:08.065320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:08.065320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:08.065320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:08.065320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:08.065320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:08.065320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:08.065320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:08.066333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:08.066333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:08.066333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:08.066333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:08.066333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:08.066333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:08.066333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:08.066333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:08.066333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:08.066333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:08.066333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:08.066333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:08.066333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:08.066333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:08.066333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:08.066333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:08.067753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:08.067753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:08.067753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:08.067753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:08.067753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:08.067753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:08.067753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:08.067753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:08.067753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:08.067753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:08.067753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:32:08.067753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:08.070297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 187/381] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:32:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:08.071554 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b5096-9088-4860-82a5-00966ead3ba5 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:08.161567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbb036-3a0e-452b-97e6-69b8c55990f5 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:08.163533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:08.228856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b5096-9088-4860-82a5-00966ead3ba5 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:08.236166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9b53b-5cbf-4050-9588-bc8767ef977c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:08.238198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9b53b-5cbf-4050-9588-bc8767ef977c None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:08.238751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9b53b-5cbf-4050-9588-bc8767ef977c None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:08.239288 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9b53b-5cbf-4050-9588-bc8767ef977c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:08.240904 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd9b53b-5cbf-4050-9588-bc8767ef977c None admin] RBAC: Adding query filter params (group.id=b58cdca6be124c00b0deece5538c211c, role.id=b4be7021e4104740b8b0b3b40ecd3d59, scope.project.id=a89544d6d9d14e759c077d7175365dca) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:08.241829 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd9b53b-5cbf-4050-9588-bc8767ef977c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:08.244179 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd9b53b-5cbf-4050-9588-bc8767ef977c None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:08.258296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 195/382] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jun 22 11:32:08 2022] GET /identity/v3/role_assignments?group.id=b58cdca6be124c00b0deece5538c211c&scope.project.id=a89544d6d9d14e759c077d7175365dca&role.id=b4be7021e4104740b8b0b3b40ecd3d59 => generated 260 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:08.259639 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9b53b-5cbf-4050-9588-bc8767ef977c None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:08.306896 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e064d-981a-4820-ba30-493cb3358e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:08.307511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e064d-981a-4820-ba30-493cb3358e63 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:08.307934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e064d-981a-4820-ba30-493cb3358e63 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:08.308321 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e064d-981a-4820-ba30-493cb3358e63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:08.309230 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e064d-981a-4820-ba30-493cb3358e63 None admin] RBAC: Adding query filter params (group.id=8521ee997e8847b6bbeb83507f355e93, role.id=96c584428d924fdba9671949f760c406, scope.project.id=c77776cd83344a69a0a4fbb7efadd00f) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:08.309548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e064d-981a-4820-ba30-493cb3358e63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:08.311105 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e064d-981a-4820-ba30-493cb3358e63 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:08.326246 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 188/383] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jun 22 11:32:08 2022] GET /identity/v3/role_assignments?group.id=8521ee997e8847b6bbeb83507f355e93&scope.project.id=c77776cd83344a69a0a4fbb7efadd00f&role.id=96c584428d924fdba9671949f760c406 => generated 260 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:08.411428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9b53b-5cbf-4050-9588-bc8767ef977c None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:08.468972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f279389b-fd15-4e7b-902c-b248d74ad515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:08.469682 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279389b-fd15-4e7b-902c-b248d74ad515 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:08.469964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279389b-fd15-4e7b-902c-b248d74ad515 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:08.470238 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f279389b-fd15-4e7b-902c-b248d74ad515 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:08.471193 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279389b-fd15-4e7b-902c-b248d74ad515 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:08.471542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279389b-fd15-4e7b-902c-b248d74ad515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:08.473156 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f279389b-fd15-4e7b-902c-b248d74ad515 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:08.490261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 196/384] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:08.497930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e064d-981a-4820-ba30-493cb3358e63 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:08.599220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:08.611834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144ed5b-a4a1-49dc-aa4b-904b36f67d81 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:08.612457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144ed5b-a4a1-49dc-aa4b-904b36f67d81 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:08.612939 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144ed5b-a4a1-49dc-aa4b-904b36f67d81 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:08.615160 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 197/385] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:08.622733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920ebd9-dc41-41d9-9ae8-d8100ae204f4 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:08.623024 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920ebd9-dc41-41d9-9ae8-d8100ae204f4 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:08.623305 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920ebd9-dc41-41d9-9ae8-d8100ae204f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:08.652869 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-5920ebd9-dc41-41d9-9ae8-d8100ae204f4 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:08.656563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-5920ebd9-dc41-41d9-9ae8-d8100ae204f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:08.705014 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:08.766822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e064d-981a-4820-ba30-493cb3358e63 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:08.821119 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 198/386] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:08.845412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef254a74-3acc-47e8-a3cf-7f131824cde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:08.846289 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef254a74-3acc-47e8-a3cf-7f131824cde0 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:08.851427 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef254a74-3acc-47e8-a3cf-7f131824cde0 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:08.851427 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef254a74-3acc-47e8-a3cf-7f131824cde0 None admin] PATH_INFO: `/v3/groups/b58cdca6be124c00b0deece5538c211c` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:08.851427 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef254a74-3acc-47e8-a3cf-7f131824cde0 None admin] RBAC: Authorizing `identity:get_group(group_id=b58cdca6be124c00b0deece5538c211c)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:08.851427 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef254a74-3acc-47e8-a3cf-7f131824cde0 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:08.856065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 189/387] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jun 22 11:32:08 2022] GET /identity/v3/groups/b58cdca6be124c00b0deece5538c211c => generated 226 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:08.861846 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920ebd9-dc41-41d9-9ae8-d8100ae204f4 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:08.875670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef254a74-3acc-47e8-a3cf-7f131824cde0 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:09.059598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef254a74-3acc-47e8-a3cf-7f131824cde0 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:09.061400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:09.109411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac622e65-6ab9-42c9-85f1-2e9b889022d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:09.110152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac622e65-6ab9-42c9-85f1-2e9b889022d2 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:09.110440 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac622e65-6ab9-42c9-85f1-2e9b889022d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:09.110719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac622e65-6ab9-42c9-85f1-2e9b889022d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:09.145888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920ebd9-dc41-41d9-9ae8-d8100ae204f4 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:09.148122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac622e65-6ab9-42c9-85f1-2e9b889022d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:09.162539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-ac622e65-6ab9-42c9-85f1-2e9b889022d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:32:09.162539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:09.162539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:09.162539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:09.162539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:09.162539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:09.162539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:09.162539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:09.162539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:09.162539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:09.162539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:09.162539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:09.162539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:09.162539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:09.162539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:09.162539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:09.163524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:09.163524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:09.163524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:09.163524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:09.163524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:09.163524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:09.163524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:09.163524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:09.163524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:09.163524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:09.163524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:09.163524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:09.163524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:09.163524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:09.163524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:09.163524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:09.164371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:09.164371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:09.164371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:09.164371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:09.164371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:09.164371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:09.164371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:09.164371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:09.164371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:09.164371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:09.164371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:32:09.164371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:09.164371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 190/388] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:09.170004 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac622e65-6ab9-42c9-85f1-2e9b889022d2 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:09.229178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41baef4-53cc-42ab-96f7-59344a205c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:09.231214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41baef4-53cc-42ab-96f7-59344a205c31 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:09.231550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41baef4-53cc-42ab-96f7-59344a205c31 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:09.231835 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41baef4-53cc-42ab-96f7-59344a205c31 None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:09.235665 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41baef4-53cc-42ab-96f7-59344a205c31 None admin] RBAC: Authorizing `identity:get_project(project_id=e263f641bede4564a5855da9ec8514ed)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:09.237148 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41baef4-53cc-42ab-96f7-59344a205c31 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:09.244681 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 199/389] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:32:08 2022] GET /identity/v3/projects/e263f641bede4564a5855da9ec8514ed => generated 298 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:09.247669 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41baef4-53cc-42ab-96f7-59344a205c31 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:09.316357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:09.410669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac622e65-6ab9-42c9-85f1-2e9b889022d2 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:09.493450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41baef4-53cc-42ab-96f7-59344a205c31 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:09.505242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bff042-cec9-45a6-9456-936d46036a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:09.505936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bff042-cec9-45a6-9456-936d46036a79 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:09.507458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bff042-cec9-45a6-9456-936d46036a79 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:09.507458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bff042-cec9-45a6-9456-936d46036a79 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:09.507458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bff042-cec9-45a6-9456-936d46036a79 None admin] RBAC: Adding query filter params (name=member) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:09.507691 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bff042-cec9-45a6-9456-936d46036a79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:09.509027 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bff042-cec9-45a6-9456-936d46036a79 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:09.531686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 191/390] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:32:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:09.536059 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bff042-cec9-45a6-9456-936d46036a79 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:09.576804 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b4283-3292-4efa-b97e-68ddd6d68a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:09.587391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b4283-3292-4efa-b97e-68ddd6d68a87 None admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:09.587391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b4283-3292-4efa-b97e-68ddd6d68a87 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:09.587391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b4283-3292-4efa-b97e-68ddd6d68a87 None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed/groups/b58cdca6be124c00b0deece5538c211c/roles/b4be7021e4104740b8b0b3b40ecd3d59` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:09.587391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b4283-3292-4efa-b97e-68ddd6d68a87 None admin] RBAC: Authorizing `identity:create_grant(project_id=e263f641bede4564a5855da9ec8514ed, group_id=b58cdca6be124c00b0deece5538c211c, role_id=b4be7021e4104740b8b0b3b40ecd3d59)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:09.587391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b4283-3292-4efa-b97e-68ddd6d68a87 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:09.593480 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:09.621557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 200/391] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jun 22 11:32:09 2022] PUT /identity/v3/projects/e263f641bede4564a5855da9ec8514ed/groups/b58cdca6be124c00b0deece5538c211c/roles/b4be7021e4104740b8b0b3b40ecd3d59 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:32:09.732284 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bff042-cec9-45a6-9456-936d46036a79 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:09.793522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a089d-6d69-48a2-9b9e-81b7dec4d045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:09.794417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a089d-6d69-48a2-9b9e-81b7dec4d045 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:09.794705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a089d-6d69-48a2-9b9e-81b7dec4d045 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:09.794985 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a089d-6d69-48a2-9b9e-81b7dec4d045 None admin] PATH_INFO: `/v3/groups/b58cdca6be124c00b0deece5538c211c` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:09.796228 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a089d-6d69-48a2-9b9e-81b7dec4d045 None admin] RBAC: Authorizing `identity:get_group(group_id=b58cdca6be124c00b0deece5538c211c)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:09.797708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a089d-6d69-48a2-9b9e-81b7dec4d045 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:09.803923 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 192/392] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jun 22 11:32:09 2022] GET /identity/v3/groups/b58cdca6be124c00b0deece5538c211c => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:09.812090 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b4283-3292-4efa-b97e-68ddd6d68a87 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:09.951380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:09.971374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b4283-3292-4efa-b97e-68ddd6d68a87 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:10.008847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6697e6-e414-4096-a916-9b91aedc6a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:10.009650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6697e6-e414-4096-a916-9b91aedc6a14 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:10.009934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6697e6-e414-4096-a916-9b91aedc6a14 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:10.010765 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6697e6-e414-4096-a916-9b91aedc6a14 None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:10.011547 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6697e6-e414-4096-a916-9b91aedc6a14 None admin] RBAC: Authorizing `identity:get_project(project_id=e263f641bede4564a5855da9ec8514ed)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:10.012985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6697e6-e414-4096-a916-9b91aedc6a14 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:10.014703 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 201/393] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:32:09 2022] GET /identity/v3/projects/e263f641bede4564a5855da9ec8514ed => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:10.021271 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a089d-6d69-48a2-9b9e-81b7dec4d045 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:10.212991 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:10.239497 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a089d-6d69-48a2-9b9e-81b7dec4d045 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:10.346352 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e93781-6f43-426f-b649-eb5a15506386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:10.347860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e93781-6f43-426f-b649-eb5a15506386 None admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:10.348396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e93781-6f43-426f-b649-eb5a15506386 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:10.348894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e93781-6f43-426f-b649-eb5a15506386 None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed/groups/b58cdca6be124c00b0deece5538c211c/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:10.352377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e93781-6f43-426f-b649-eb5a15506386 None admin] RBAC: Authorizing `identity:create_grant(project_id=e263f641bede4564a5855da9ec8514ed, group_id=b58cdca6be124c00b0deece5538c211c, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:10.355079 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e93781-6f43-426f-b649-eb5a15506386 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:10.392219 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 193/394] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jun 22 11:32:10 2022] PUT /identity/v3/projects/e263f641bede4564a5855da9ec8514ed/groups/b58cdca6be124c00b0deece5538c211c/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:32:10.585438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72510e-297b-4e94-957b-19d7cc2095f4 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:10.585972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72510e-297b-4e94-957b-19d7cc2095f4 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:10.591380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72510e-297b-4e94-957b-19d7cc2095f4 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:10.591587 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 202/395] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:10.600096 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa4ae1-a2b7-4303-b26c-f67d515bcd9e None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:10.600565 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa4ae1-a2b7-4303-b26c-f67d515bcd9e None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:10.601015 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa4ae1-a2b7-4303-b26c-f67d515bcd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:10.650770 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-6bfa4ae1-a2b7-4303-b26c-f67d515bcd9e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:10.652827 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-6bfa4ae1-a2b7-4303-b26c-f67d515bcd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:10.800365 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 194/396] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:10.875845 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72510e-297b-4e94-957b-19d7cc2095f4 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:10.976480 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa18954-b7da-4217-bc0d-d05439a13ea3 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:10.977116 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa18954-b7da-4217-bc0d-d05439a13ea3 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:10.977585 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa18954-b7da-4217-bc0d-d05439a13ea3 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:10.983503 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 195/397] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:10.985118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d62d93-0d50-4f03-9dd1-d5afb780a9f9 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:10.985619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d62d93-0d50-4f03-9dd1-d5afb780a9f9 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:10.986088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d62d93-0d50-4f03-9dd1-d5afb780a9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:11.031398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-c9d62d93-0d50-4f03-9dd1-d5afb780a9f9 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:11.033247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-c9d62d93-0d50-4f03-9dd1-d5afb780a9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:11.137937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72510e-297b-4e94-957b-19d7cc2095f4 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:11.247395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 196/398] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 22 11:32:11.282087 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e8d9c-3244-40c8-aa77-388eeb302e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:11.283367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e8d9c-3244-40c8-aa77-388eeb302e7e None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:11.283855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e8d9c-3244-40c8-aa77-388eeb302e7e None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:11.284344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e8d9c-3244-40c8-aa77-388eeb302e7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:11.315050 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e8d9c-3244-40c8-aa77-388eeb302e7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:11.320084 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d62d93-0d50-4f03-9dd1-d5afb780a9f9 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:11.345189 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:11.346619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-b42e8d9c-3244-40c8-aa77-388eeb302e7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:32:11.346619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:11.346619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:11.346619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:11.346619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:11.346619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:11.346619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:11.346619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:11.346619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:11.346619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:11.346619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:11.346619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:11.346619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:11.346619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:11.346619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:11.346619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:11.347528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:11.347528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:11.347528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:11.347528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:11.347528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:11.347528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:11.347528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:11.347528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:11.347528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:11.347528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:11.347528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:11.347528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:11.347528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:11.347528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:11.347528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:11.347528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:11.348952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:11.348952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:11.348952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:11.348952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:11.348952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:11.348952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:11.348952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:11.348952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:11.348952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:11.348952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:11.348952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:32:11.348952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:11.355669 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 203/399] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 480 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:11.366923 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e8d9c-3244-40c8-aa77-388eeb302e7e None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:11.428159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:11.586726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d62d93-0d50-4f03-9dd1-d5afb780a9f9 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:11.690632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e8d9c-3244-40c8-aa77-388eeb302e7e None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:11.709277 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c2096-d904-45e7-9240-7ceb22466100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:11.710964 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c2096-d904-45e7-9240-7ceb22466100 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:11.712132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c2096-d904-45e7-9240-7ceb22466100 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:11.712697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c2096-d904-45e7-9240-7ceb22466100 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:11.746533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656e984-95cd-47c1-a70d-80765e414a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:11.748901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c2096-d904-45e7-9240-7ceb22466100 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:11.752169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656e984-95cd-47c1-a70d-80765e414a5f None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:11.752635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656e984-95cd-47c1-a70d-80765e414a5f None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:11.753041 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656e984-95cd-47c1-a70d-80765e414a5f None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:11.753965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656e984-95cd-47c1-a70d-80765e414a5f None admin] RBAC: Adding query filter params (name=member) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:11.754979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656e984-95cd-47c1-a70d-80765e414a5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:11.756800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656e984-95cd-47c1-a70d-80765e414a5f None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:11.766581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-501c2096-d904-45e7-9240-7ceb22466100 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:32:11.766581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:11.766581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:11.766581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:11.766581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:11.766581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:11.766581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:11.766581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:11.766581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:11.766581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:11.766581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:11.766581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:11.766581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:11.766581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:11.766581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:11.766581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:11.771333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:11.771333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:11.771333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:11.771333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:11.771333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:11.771333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:11.771333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:11.771333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:11.771333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:11.771333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:11.771333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:11.771333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:11.771333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:11.771333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:11.771333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:11.771333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:11.776885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:11.776885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:11.776885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:11.776885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:11.776885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:11.776885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:11.776885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:11.776885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:11.776885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:11.776885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:11.776885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:32:11.776885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:11.777736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 204/400] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:32:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:11.777736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2b817-18cd-4947-a9f3-74fbda647db5 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:11.777736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2b817-18cd-4947-a9f3-74fbda647db5 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:11.777736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2b817-18cd-4947-a9f3-74fbda647db5 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:11.777736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 205/401] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:11.780341 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2b817-18cd-4947-a9f3-74fbda647db5 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:11.786696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 197/402] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:32:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 462 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:32:11.790727 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62080c-1719-40d9-806a-32171fb885d5 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:11.791238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62080c-1719-40d9-806a-32171fb885d5 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:11.791709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62080c-1719-40d9-806a-32171fb885d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:11.830150 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-9a62080c-1719-40d9-806a-32171fb885d5 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:11.831654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-9a62080c-1719-40d9-806a-32171fb885d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:11.971965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:11.990247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 198/403] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:11.991551 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62080c-1719-40d9-806a-32171fb885d5 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:12.001980 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:12.005810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2b817-18cd-4947-a9f3-74fbda647db5 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:12.057254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f63b81-e272-40bf-8132-4d4433080eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:12.058373 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f63b81-e272-40bf-8132-4d4433080eb8 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:12.058723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f63b81-e272-40bf-8132-4d4433080eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:12.059071 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f63b81-e272-40bf-8132-4d4433080eb8 None admin] PATH_INFO: `/v3/groups/b58cdca6be124c00b0deece5538c211c` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:12.060420 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f63b81-e272-40bf-8132-4d4433080eb8 None admin] RBAC: Authorizing `identity:get_group(group_id=b58cdca6be124c00b0deece5538c211c)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:12.063377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f63b81-e272-40bf-8132-4d4433080eb8 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:12.071391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 206/404] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jun 22 11:32:11 2022] GET /identity/v3/groups/b58cdca6be124c00b0deece5538c211c => generated 226 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:12.071391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f63b81-e272-40bf-8132-4d4433080eb8 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:12.151926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62080c-1719-40d9-806a-32171fb885d5 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:12.208834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f63b81-e272-40bf-8132-4d4433080eb8 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:12.215300 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00c0c2-1cf2-4123-80b8-41c014b7f182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:12.216046 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00c0c2-1cf2-4123-80b8-41c014b7f182 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:12.216326 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00c0c2-1cf2-4123-80b8-41c014b7f182 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:12.216602 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00c0c2-1cf2-4123-80b8-41c014b7f182 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:12.217363 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00c0c2-1cf2-4123-80b8-41c014b7f182 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:12.217669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00c0c2-1cf2-4123-80b8-41c014b7f182 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:12.219069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00c0c2-1cf2-4123-80b8-41c014b7f182 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:12.235479 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 199/405] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:12.235479 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00c0c2-1cf2-4123-80b8-41c014b7f182 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:12.248875 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1638db1-be4e-4ee5-b6f4-898079235d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:12.249690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1638db1-be4e-4ee5-b6f4-898079235d3a None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:12.250043 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1638db1-be4e-4ee5-b6f4-898079235d3a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:12.250388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1638db1-be4e-4ee5-b6f4-898079235d3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:12.262634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1638db1-be4e-4ee5-b6f4-898079235d3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:12.272664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-b1638db1-be4e-4ee5-b6f4-898079235d3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:32:12.272664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:12.272664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:12.272664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:12.272664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:12.272664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:12.272664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:12.272664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:12.272664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:12.272664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:12.272664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:12.272664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:12.272664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:12.272664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:12.272664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:12.272664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:12.273699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:12.273699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:12.273699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:12.273699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:12.273699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:12.273699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:12.273699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:12.273699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:12.273699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:12.273699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:12.273699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:12.273699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:12.273699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:12.273699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:12.273699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:12.273699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:12.274555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:12.274555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:12.274555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:12.274555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:12.274555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:12.274555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:12.274555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:12.274555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:12.274555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:12.274555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:12.274555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:32:12.274555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:12.274555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 207/406] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:32:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:12.274555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1638db1-be4e-4ee5-b6f4-898079235d3a None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:12.282900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:12.323782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:12.376758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00c0c2-1cf2-4123-80b8-41c014b7f182 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:12.419477 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1638db1-be4e-4ee5-b6f4-898079235d3a None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:12.433427 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347cb04-31cd-49ae-852b-240f45da79bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:12.434667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347cb04-31cd-49ae-852b-240f45da79bc None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:12.435149 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347cb04-31cd-49ae-852b-240f45da79bc None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:12.435663 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347cb04-31cd-49ae-852b-240f45da79bc None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:12.438520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347cb04-31cd-49ae-852b-240f45da79bc None admin] RBAC: Authorizing `identity:get_project(project_id=a89544d6d9d14e759c077d7175365dca)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:12.442187 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347cb04-31cd-49ae-852b-240f45da79bc None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:12.449429 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 200/407] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:32:12 2022] GET /identity/v3/projects/a89544d6d9d14e759c077d7175365dca => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:12.450575 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347cb04-31cd-49ae-852b-240f45da79bc None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:12.469362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e460a8-41e6-4f65-a59e-e5ed8cd98cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:12.471210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e460a8-41e6-4f65-a59e-e5ed8cd98cec None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:12.471210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e460a8-41e6-4f65-a59e-e5ed8cd98cec None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:12.471210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e460a8-41e6-4f65-a59e-e5ed8cd98cec None admin] PATH_INFO: `/v3/groups/8521ee997e8847b6bbeb83507f355e93` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:12.475672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e460a8-41e6-4f65-a59e-e5ed8cd98cec None admin] RBAC: Authorizing `identity:get_group(group_id=8521ee997e8847b6bbeb83507f355e93)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:12.477155 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e460a8-41e6-4f65-a59e-e5ed8cd98cec None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:12.480197 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 208/408] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jun 22 11:32:12 2022] GET /identity/v3/groups/8521ee997e8847b6bbeb83507f355e93 => generated 229 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:12.481954 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e460a8-41e6-4f65-a59e-e5ed8cd98cec None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:12.575898 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347cb04-31cd-49ae-852b-240f45da79bc None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:12.584935 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:12.614647 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:12.622011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513c68f-8d36-4929-9146-d4acfb3d74f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:12.622941 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513c68f-8d36-4929-9146-d4acfb3d74f9 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:12.623429 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513c68f-8d36-4929-9146-d4acfb3d74f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:12.623864 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513c68f-8d36-4929-9146-d4acfb3d74f9 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:12.624785 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513c68f-8d36-4929-9146-d4acfb3d74f9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:12.625236 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513c68f-8d36-4929-9146-d4acfb3d74f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:12.626970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513c68f-8d36-4929-9146-d4acfb3d74f9 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:12.629350 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e460a8-41e6-4f65-a59e-e5ed8cd98cec None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:12.650929 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 201/409] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:12.650929 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513c68f-8d36-4929-9146-d4acfb3d74f9 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:12.683374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41f26c-7ff2-44c1-9f99-9724b0a31f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:12.684355 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41f26c-7ff2-44c1-9f99-9724b0a31f55 None admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:12.684642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41f26c-7ff2-44c1-9f99-9724b0a31f55 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:12.684920 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41f26c-7ff2-44c1-9f99-9724b0a31f55 None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca/groups/b58cdca6be124c00b0deece5538c211c/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:12.687420 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41f26c-7ff2-44c1-9f99-9724b0a31f55 None admin] RBAC: Authorizing `identity:create_grant(project_id=a89544d6d9d14e759c077d7175365dca, group_id=b58cdca6be124c00b0deece5538c211c, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:12.688933 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41f26c-7ff2-44c1-9f99-9724b0a31f55 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:12.705431 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 209/410] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jun 22 11:32:12 2022] PUT /identity/v3/projects/a89544d6d9d14e759c077d7175365dca/groups/b58cdca6be124c00b0deece5538c211c/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:32:12.706862 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e2d01-66e6-46dd-811a-59208fe494ea None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:12.707150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e2d01-66e6-46dd-811a-59208fe494ea None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:12.707490 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e2d01-66e6-46dd-811a-59208fe494ea None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:12.709089 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 210/411] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:12.710183 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e2d01-66e6-46dd-811a-59208fe494ea None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:12.822823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513c68f-8d36-4929-9146-d4acfb3d74f9 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:12.893675 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e2d01-66e6-46dd-811a-59208fe494ea None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:12.899288 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499dcd79-6abc-478b-abd0-a4e2d0f5dc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:12.900212 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499dcd79-6abc-478b-abd0-a4e2d0f5dc24 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:12.900497 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499dcd79-6abc-478b-abd0-a4e2d0f5dc24 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:12.900772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499dcd79-6abc-478b-abd0-a4e2d0f5dc24 None admin] PATH_INFO: `/v3/projects/c77776cd83344a69a0a4fbb7efadd00f` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:12.901990 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499dcd79-6abc-478b-abd0-a4e2d0f5dc24 None admin] RBAC: Authorizing `identity:get_project(project_id=c77776cd83344a69a0a4fbb7efadd00f)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:12.903750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499dcd79-6abc-478b-abd0-a4e2d0f5dc24 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:12.906049 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 202/412] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:32:12 2022] GET /identity/v3/projects/c77776cd83344a69a0a4fbb7efadd00f => generated 344 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:12.912772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50a112-41bc-43a2-a81f-72dbe251ab87 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:12.913097 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50a112-41bc-43a2-a81f-72dbe251ab87 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:12.913382 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50a112-41bc-43a2-a81f-72dbe251ab87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:12.938148 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-dd50a112-41bc-43a2-a81f-72dbe251ab87 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:12.943569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-dd50a112-41bc-43a2-a81f-72dbe251ab87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:12.953246 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:12.959597 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2390d9-e3c8-47c6-88fe-9e24807c70b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:12.963606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2390d9-e3c8-47c6-88fe-9e24807c70b3 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:12.964089 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2390d9-e3c8-47c6-88fe-9e24807c70b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:12.964514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2390d9-e3c8-47c6-88fe-9e24807c70b3 None admin] PATH_INFO: `/v3/groups/b58cdca6be124c00b0deece5538c211c` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:12.966992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2390d9-e3c8-47c6-88fe-9e24807c70b3 None admin] RBAC: Authorizing `identity:get_group(group_id=b58cdca6be124c00b0deece5538c211c)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:12.967694 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2390d9-e3c8-47c6-88fe-9e24807c70b3 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:12.971466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 211/413] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jun 22 11:32:12 2022] GET /identity/v3/groups/b58cdca6be124c00b0deece5538c211c => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:12.972197 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2390d9-e3c8-47c6-88fe-9e24807c70b3 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:13.067769 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:13.091708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 203/414] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:13.092873 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50a112-41bc-43a2-a81f-72dbe251ab87 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:13.142904 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2390d9-e3c8-47c6-88fe-9e24807c70b3 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:13.200123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97cdb20-2983-4084-a1c4-e78f5aa56a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:13.201074 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cdb20-2983-4084-a1c4-e78f5aa56a1d None admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:13.201491 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cdb20-2983-4084-a1c4-e78f5aa56a1d None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:13.201772 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cdb20-2983-4084-a1c4-e78f5aa56a1d None admin] PATH_INFO: `/v3/projects/c77776cd83344a69a0a4fbb7efadd00f/groups/8521ee997e8847b6bbeb83507f355e93/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:13.205985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97cdb20-2983-4084-a1c4-e78f5aa56a1d None admin] RBAC: Authorizing `identity:create_grant(project_id=c77776cd83344a69a0a4fbb7efadd00f, group_id=8521ee997e8847b6bbeb83507f355e93, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:13.208102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97cdb20-2983-4084-a1c4-e78f5aa56a1d None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:13.227321 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50a112-41bc-43a2-a81f-72dbe251ab87 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:13.235839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 212/415] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jun 22 11:32:12 2022] PUT /identity/v3/projects/c77776cd83344a69a0a4fbb7efadd00f/groups/8521ee997e8847b6bbeb83507f355e93/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:32:13.235839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb55415-7277-4ef7-a7d6-0a631e2c11fc None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:13.235839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb55415-7277-4ef7-a7d6-0a631e2c11fc None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:13.235839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb55415-7277-4ef7-a7d6-0a631e2c11fc None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:13.238771 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 213/416] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:13.238771 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb55415-7277-4ef7-a7d6-0a631e2c11fc None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:13.296385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73bf35e-a5f1-4d26-93b5-f3be46960c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:13.297377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bf35e-a5f1-4d26-93b5-f3be46960c4d None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:13.297648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bf35e-a5f1-4d26-93b5-f3be46960c4d None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:13.298271 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bf35e-a5f1-4d26-93b5-f3be46960c4d None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:13.300634 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73bf35e-a5f1-4d26-93b5-f3be46960c4d None admin] RBAC: Authorizing `identity:get_project(project_id=a89544d6d9d14e759c077d7175365dca)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:13.308124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73bf35e-a5f1-4d26-93b5-f3be46960c4d None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:13.318259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 204/417] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:32:13 2022] GET /identity/v3/projects/a89544d6d9d14e759c077d7175365dca => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:13.318259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6cb574-e201-4603-9d85-e2e351d2ebbb None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:13.318259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6cb574-e201-4603-9d85-e2e351d2ebbb None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:13.318259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6cb574-e201-4603-9d85-e2e351d2ebbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:13.356371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-6f6cb574-e201-4603-9d85-e2e351d2ebbb None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:13.358474 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-6f6cb574-e201-4603-9d85-e2e351d2ebbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:13.361432 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:13.465748 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb55415-7277-4ef7-a7d6-0a631e2c11fc None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:13.481313 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:13.509240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 205/418] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:13.510308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6cb574-e201-4603-9d85-e2e351d2ebbb None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:13.547296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ab48d-dfaa-46ab-ac5f-37dd8db85836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:13.549294 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ab48d-dfaa-46ab-ac5f-37dd8db85836 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:13.549544 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ab48d-dfaa-46ab-ac5f-37dd8db85836 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:13.549816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ab48d-dfaa-46ab-ac5f-37dd8db85836 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:13.576899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ab48d-dfaa-46ab-ac5f-37dd8db85836 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:13.585714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-0a5ab48d-dfaa-46ab-ac5f-37dd8db85836 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:32:13.585714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:13.585714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:13.585714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:13.585714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:13.585714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:13.585714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:13.585714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:13.585714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:13.585714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:13.585714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:13.585714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:13.585714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:13.585714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:13.585714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:13.585714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:13.586647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:13.586647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:13.586647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:13.586647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:13.586647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:13.586647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:13.586647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:13.586647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:13.586647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:13.586647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:13.586647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:13.586647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:13.586647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:13.586647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:13.586647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:13.586647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:13.587506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:13.587506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:13.587506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:13.587506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:13.587506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:13.587506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:13.587506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:13.587506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:13.587506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:13.587506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:13.587506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:32:13.587506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:13.596948 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 214/419] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:32:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:13.596948 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ab48d-dfaa-46ab-ac5f-37dd8db85836 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:13.735795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ab48d-dfaa-46ab-ac5f-37dd8db85836 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:13.744848 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6cb574-e201-4603-9d85-e2e351d2ebbb None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:13.766528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:13.785064 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4da9d4-5f42-4beb-9261-b7670a25f3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:13.786771 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4da9d4-5f42-4beb-9261-b7670a25f3bd None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:13.787446 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4da9d4-5f42-4beb-9261-b7670a25f3bd None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:13.788049 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4da9d4-5f42-4beb-9261-b7670a25f3bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:13.804232 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4da9d4-5f42-4beb-9261-b7670a25f3bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:13.811469 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d8f82-eacd-4d06-bfb0-2901e1fc6ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:13.812499 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d8f82-eacd-4d06-bfb0-2901e1fc6ba8 None admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:13.812886 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d8f82-eacd-4d06-bfb0-2901e1fc6ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:13.813264 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d8f82-eacd-4d06-bfb0-2901e1fc6ba8 None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca/groups/b58cdca6be124c00b0deece5538c211c/roles/b4be7021e4104740b8b0b3b40ecd3d59` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:13.815502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-7e4da9d4-5f42-4beb-9261-b7670a25f3bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:32:13.815502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:13.815502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:13.815502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:13.815502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:13.815502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:13.815502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:13.815502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:13.815502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:13.815502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:13.815502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:13.815502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:13.815502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:13.815502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:13.815502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:13.815502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:13.816736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:13.816736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:13.816736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:13.816736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:13.816736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:13.816736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:13.816736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:13.816736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:13.816736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:13.816736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:13.816736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:13.816736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:13.816736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:13.816736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:13.816736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:13.816736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:13.818284 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:13.818284 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:13.818284 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:13.818284 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:13.818284 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:13.818284 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:13.818284 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:13.818284 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:13.818284 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:13.818284 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:13.818284 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:32:13.818284 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:13.820958 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 215/420] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:13.821939 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d8f82-eacd-4d06-bfb0-2901e1fc6ba8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a89544d6d9d14e759c077d7175365dca, group_id=b58cdca6be124c00b0deece5538c211c, role_id=b4be7021e4104740b8b0b3b40ecd3d59)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:13.822228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4da9d4-5f42-4beb-9261-b7670a25f3bd None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:13.823682 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d8f82-eacd-4d06-bfb0-2901e1fc6ba8 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:13.844617 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 206/421] 10.222.0.36 () {58 vars in 1442 bytes} [Wed Jun 22 11:32:13 2022] PUT /identity/v3/projects/a89544d6d9d14e759c077d7175365dca/groups/b58cdca6be124c00b0deece5538c211c/roles/b4be7021e4104740b8b0b3b40ecd3d59 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:32:13.847196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d8f82-eacd-4d06-bfb0-2901e1fc6ba8 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:13.935621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:14.013494 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d8f82-eacd-4d06-bfb0-2901e1fc6ba8 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:14.017965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4da9d4-5f42-4beb-9261-b7670a25f3bd None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:14.059750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fe4e0-cbe5-4c1f-9c80-b08b6fac7f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:14.061105 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fe4e0-cbe5-4c1f-9c80-b08b6fac7f88 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:14.061387 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fe4e0-cbe5-4c1f-9c80-b08b6fac7f88 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:14.061718 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fe4e0-cbe5-4c1f-9c80-b08b6fac7f88 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:14.062555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fe4e0-cbe5-4c1f-9c80-b08b6fac7f88 None admin] RBAC: Adding query filter params (name=member) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:14.062864 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fe4e0-cbe5-4c1f-9c80-b08b6fac7f88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:14.067379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fe4e0-cbe5-4c1f-9c80-b08b6fac7f88 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:14.075803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 207/422] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:32:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:14.088667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fe4e0-cbe5-4c1f-9c80-b08b6fac7f88 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:14.136142 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a74051-2e1f-46a0-a9a9-01b11c913fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:14.137046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a74051-2e1f-46a0-a9a9-01b11c913fd5 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:14.137332 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a74051-2e1f-46a0-a9a9-01b11c913fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:14.137613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a74051-2e1f-46a0-a9a9-01b11c913fd5 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:14.138390 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a74051-2e1f-46a0-a9a9-01b11c913fd5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:14.138699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a74051-2e1f-46a0-a9a9-01b11c913fd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:14.140916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a74051-2e1f-46a0-a9a9-01b11c913fd5 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:14.165624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 216/423] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:14.180573 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a74051-2e1f-46a0-a9a9-01b11c913fd5 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:14.236650 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fe4e0-cbe5-4c1f-9c80-b08b6fac7f88 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:14.242550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:14.254531 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:14.280353 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f562cc0-92df-4caa-854f-ae05aae8c9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:14.281664 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f562cc0-92df-4caa-854f-ae05aae8c9d7 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:14.282170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f562cc0-92df-4caa-854f-ae05aae8c9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:14.282644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f562cc0-92df-4caa-854f-ae05aae8c9d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:14.378738 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a74051-2e1f-46a0-a9a9-01b11c913fd5 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:14.406902 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f562cc0-92df-4caa-854f-ae05aae8c9d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:14.410930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f562cc0-92df-4caa-854f-ae05aae8c9d7 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:14.452279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9ed5c-82d6-45db-9572-cbd6208e0918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:14.452952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9ed5c-82d6-45db-9572-cbd6208e0918 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:14.453242 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9ed5c-82d6-45db-9572-cbd6208e0918 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:14.453526 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9ed5c-82d6-45db-9572-cbd6208e0918 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:14.557661 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 208/424] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 22 11:32:14.565780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f562cc0-92df-4caa-854f-ae05aae8c9d7 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:14.587577 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:14.667316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9ed5c-82d6-45db-9572-cbd6208e0918 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:14.669850 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9ed5c-82d6-45db-9572-cbd6208e0918 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:14.703043 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f562cc0-92df-4caa-854f-ae05aae8c9d7 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:14.750856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54309652-2454-4401-abe0-da8455c5a25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:14.752141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54309652-2454-4401-abe0-da8455c5a25b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:14.752626 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54309652-2454-4401-abe0-da8455c5a25b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:14.753102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54309652-2454-4401-abe0-da8455c5a25b None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:14.754699 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54309652-2454-4401-abe0-da8455c5a25b None admin] RBAC: Authorizing `identity:get_project(project_id=e263f641bede4564a5855da9ec8514ed)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:14.756578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54309652-2454-4401-abe0-da8455c5a25b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:14.761687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 209/425] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:32:14 2022] GET /identity/v3/projects/e263f641bede4564a5855da9ec8514ed => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:14.767633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54309652-2454-4401-abe0-da8455c5a25b None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:14.805297 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:14.943616 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 217/426] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 764 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:14.951304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9ed5c-82d6-45db-9572-cbd6208e0918 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:14.956371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:15.001248 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54309652-2454-4401-abe0-da8455c5a25b None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:15.076479 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fc260-b005-4a2f-8f7e-838478f22d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:15.077765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fc260-b005-4a2f-8f7e-838478f22d62 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:15.078664 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fc260-b005-4a2f-8f7e-838478f22d62 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:15.079374 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fc260-b005-4a2f-8f7e-838478f22d62 None admin] PATH_INFO: `/v3/groups/b58cdca6be124c00b0deece5538c211c` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:15.081724 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8fc260-b005-4a2f-8f7e-838478f22d62 None admin] RBAC: Authorizing `identity:get_group(group_id=b58cdca6be124c00b0deece5538c211c)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:15.084366 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8fc260-b005-4a2f-8f7e-838478f22d62 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:15.088331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 210/427] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jun 22 11:32:14 2022] GET /identity/v3/groups/b58cdca6be124c00b0deece5538c211c => generated 226 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:15.094054 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fc260-b005-4a2f-8f7e-838478f22d62 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:15.181724 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9ed5c-82d6-45db-9572-cbd6208e0918 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:15.289908 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fc260-b005-4a2f-8f7e-838478f22d62 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:15.296852 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec0461-e5a3-4790-a907-480ffc231ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:15.297742 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec0461-e5a3-4790-a907-480ffc231ca7 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:15.298024 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec0461-e5a3-4790-a907-480ffc231ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:15.298300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec0461-e5a3-4790-a907-480ffc231ca7 None admin] PATH_INFO: `/v3/projects/e263f641bede4564a5855da9ec8514ed` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:15.303094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec0461-e5a3-4790-a907-480ffc231ca7 None admin] RBAC: Authorizing `identity:get_project(project_id=e263f641bede4564a5855da9ec8514ed)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:15.303094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec0461-e5a3-4790-a907-480ffc231ca7 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:15.315859 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 218/428] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:32:14 2022] GET /identity/v3/projects/e263f641bede4564a5855da9ec8514ed => generated 298 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:15.321977 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec0461-e5a3-4790-a907-480ffc231ca7 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:15.338263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462beb8c-d9a8-4261-8c67-3468b7611551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:15.339335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462beb8c-d9a8-4261-8c67-3468b7611551 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:15.339871 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462beb8c-d9a8-4261-8c67-3468b7611551 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:15.340397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462beb8c-d9a8-4261-8c67-3468b7611551 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:15.341779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462beb8c-d9a8-4261-8c67-3468b7611551 None admin] RBAC: Adding query filter params (group.id=b58cdca6be124c00b0deece5538c211c, role.id=f2336919c5104f6a8210e99994f4d455, scope.project.id=e263f641bede4564a5855da9ec8514ed) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:15.342430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462beb8c-d9a8-4261-8c67-3468b7611551 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:15.344222 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462beb8c-d9a8-4261-8c67-3468b7611551 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:15.356073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:15.357627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 211/429] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jun 22 11:32:15 2022] GET /identity/v3/role_assignments?group.id=b58cdca6be124c00b0deece5538c211c&scope.project.id=e263f641bede4564a5855da9ec8514ed&role.id=f2336919c5104f6a8210e99994f4d455 => generated 611 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:15.393277 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254406f-f671-4789-81bd-c7d09ac46c57 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:15.393905 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254406f-f671-4789-81bd-c7d09ac46c57 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:15.394385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254406f-f671-4789-81bd-c7d09ac46c57 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:15.397023 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 212/430] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:32:15.415928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e916c73-aaf3-4876-932f-f4dc494fecd7 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:15.418508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e916c73-aaf3-4876-932f-f4dc494fecd7 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:15.419684 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e916c73-aaf3-4876-932f-f4dc494fecd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:15.463041 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-5e916c73-aaf3-4876-932f-f4dc494fecd7 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:15.474106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5e916c73-aaf3-4876-932f-f4dc494fecd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:15.506612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:15.627454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec0461-e5a3-4790-a907-480ffc231ca7 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:15.643771 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 213/431] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:15.671603 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa210e-f2c5-4b66-8e96-21a6217fec13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:15.672476 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa210e-f2c5-4b66-8e96-21a6217fec13 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:15.672759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa210e-f2c5-4b66-8e96-21a6217fec13 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:15.673036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa210e-f2c5-4b66-8e96-21a6217fec13 None admin] PATH_INFO: `/v3/groups/b58cdca6be124c00b0deece5538c211c` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:15.674742 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa210e-f2c5-4b66-8e96-21a6217fec13 None admin] RBAC: Authorizing `identity:get_group(group_id=b58cdca6be124c00b0deece5538c211c)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:15.676320 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa210e-f2c5-4b66-8e96-21a6217fec13 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:15.678334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 219/432] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jun 22 11:32:15 2022] GET /identity/v3/groups/b58cdca6be124c00b0deece5538c211c => generated 226 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:15.684487 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e916c73-aaf3-4876-932f-f4dc494fecd7 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:15.701149 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa210e-f2c5-4b66-8e96-21a6217fec13 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:15.804690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:15.838388 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e916c73-aaf3-4876-932f-f4dc494fecd7 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:15.845418 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa210e-f2c5-4b66-8e96-21a6217fec13 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:15.891551 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6389a71-1cfd-47c1-a59b-084c8b86c28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:15.891971 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6389a71-1cfd-47c1-a59b-084c8b86c28f None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:15.892260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6389a71-1cfd-47c1-a59b-084c8b86c28f None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:15.892619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6389a71-1cfd-47c1-a59b-084c8b86c28f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:15.893920 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e5728-585f-45c2-9458-18da1a73fc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:15.894591 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e5728-585f-45c2-9458-18da1a73fc36 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:15.894927 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e5728-585f-45c2-9458-18da1a73fc36 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:15.895548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e5728-585f-45c2-9458-18da1a73fc36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:15.898226 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e5728-585f-45c2-9458-18da1a73fc36 None admin] RBAC: Adding query filter params (group.id=b58cdca6be124c00b0deece5538c211c, role.id=b4be7021e4104740b8b0b3b40ecd3d59, scope.project.id=e263f641bede4564a5855da9ec8514ed) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:15.898226 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e5728-585f-45c2-9458-18da1a73fc36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:15.902370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e5728-585f-45c2-9458-18da1a73fc36 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:15.915022 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6389a71-1cfd-47c1-a59b-084c8b86c28f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:15.917584 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 214/433] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jun 22 11:32:15 2022] GET /identity/v3/role_assignments?group.id=b58cdca6be124c00b0deece5538c211c&scope.project.id=e263f641bede4564a5855da9ec8514ed&role.id=b4be7021e4104740b8b0b3b40ecd3d59 => generated 611 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:15.929942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-e6389a71-1cfd-47c1-a59b-084c8b86c28f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:32:15.929942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:15.929942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:15.929942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:15.929942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:15.929942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:15.929942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:15.929942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:15.929942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:15.929942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:15.929942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:15.929942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:15.929942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:15.929942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:15.929942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:15.929942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:15.930899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:15.930899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:15.930899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:15.930899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:15.930899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:15.930899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:15.930899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:15.930899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:15.930899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:15.930899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:15.930899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:15.930899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:15.930899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:15.930899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:15.930899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:15.930899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:15.931699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:15.931699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:15.931699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:15.931699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:15.931699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:15.931699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:15.931699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:15.931699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:15.931699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:15.931699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:15.931699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 22 11:32:15.931699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:15.931699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 220/434] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:15.941258 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e5728-585f-45c2-9458-18da1a73fc36 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:15.962295 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:15.997115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc74e8b-0209-4bfc-a948-2bc34519cdd8 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:15.997764 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc74e8b-0209-4bfc-a948-2bc34519cdd8 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:15.998245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc74e8b-0209-4bfc-a948-2bc34519cdd8 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:16.003380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 221/435] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:16.011412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d207158-3e02-4740-9026-28ff181d4e0a None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:16.011412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d207158-3e02-4740-9026-28ff181d4e0a None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:16.011412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d207158-3e02-4740-9026-28ff181d4e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:16.048833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-6d207158-3e02-4740-9026-28ff181d4e0a None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:16.048833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-6d207158-3e02-4740-9026-28ff181d4e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:16.095457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e5728-585f-45c2-9458-18da1a73fc36 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:16.141843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88c99d-bc89-4542-b67a-29d48899b401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:16.142883 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88c99d-bc89-4542-b67a-29d48899b401 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:16.143404 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88c99d-bc89-4542-b67a-29d48899b401 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:16.143918 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88c99d-bc89-4542-b67a-29d48899b401 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:16.145439 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88c99d-bc89-4542-b67a-29d48899b401 None admin] RBAC: Adding query filter params (name=member) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:16.145968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88c99d-bc89-4542-b67a-29d48899b401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:16.147621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88c99d-bc89-4542-b67a-29d48899b401 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:16.159932 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 215/436] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:32:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:16.166286 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88c99d-bc89-4542-b67a-29d48899b401 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:16.200090 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 222/437] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:16.224280 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d207158-3e02-4740-9026-28ff181d4e0a None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:16.302287 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:16.351639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88c99d-bc89-4542-b67a-29d48899b401 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:16.359709 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:16.404589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d207158-3e02-4740-9026-28ff181d4e0a None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:16.409223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ae62c-a994-4ba4-89f4-8109780b00ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:16.409223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ae62c-a994-4ba4-89f4-8109780b00ca None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:16.409223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ae62c-a994-4ba4-89f4-8109780b00ca None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:16.409223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ae62c-a994-4ba4-89f4-8109780b00ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:16.458718 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44407f6-e014-4948-99b5-f75c36edcd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:16.459565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44407f6-e014-4948-99b5-f75c36edcd79 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:16.459847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44407f6-e014-4948-99b5-f75c36edcd79 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:16.460121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44407f6-e014-4948-99b5-f75c36edcd79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:16.477163 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44407f6-e014-4948-99b5-f75c36edcd79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:16.487938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-f44407f6-e014-4948-99b5-f75c36edcd79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:32:16.487938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:16.487938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:16.487938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:16.487938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:16.487938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:16.487938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:16.487938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:16.487938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:16.487938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:16.487938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:16.487938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:16.487938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:16.487938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:16.487938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:16.487938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:16.489151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:16.489151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:16.489151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:16.489151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:16.489151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:16.489151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:16.489151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:16.489151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:16.489151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:16.489151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:16.489151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:16.489151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:16.489151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:16.489151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:16.489151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:16.489151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:16.490150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:16.490150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:16.490150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:16.490150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:16.490150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:16.490150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:16.490150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:16.490150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:16.490150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:16.490150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:16.490150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:32:16.490150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:16.494701 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 223/438] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:32:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:16.500657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44407f6-e014-4948-99b5-f75c36edcd79 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:16.532599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ae62c-a994-4ba4-89f4-8109780b00ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:16.534322 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ae62c-a994-4ba4-89f4-8109780b00ca None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:16.614409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:16.630308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44407f6-e014-4948-99b5-f75c36edcd79 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:16.666549 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 216/439] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:16.668083 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a05b0-7cab-437f-a6ca-bbf4d09b060a None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:16.668375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a05b0-7cab-437f-a6ca-bbf4d09b060a None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:16.669322 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a05b0-7cab-437f-a6ca-bbf4d09b060a None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:16.675089 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 217/440] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:16.675089 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a05b0-7cab-437f-a6ca-bbf4d09b060a None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:16.675324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:16.687465 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ceb7f-a5e9-454b-9937-ad3b37a00b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:16.688574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ceb7f-a5e9-454b-9937-ad3b37a00b8f None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:16.689078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ceb7f-a5e9-454b-9937-ad3b37a00b8f None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:16.689577 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ceb7f-a5e9-454b-9937-ad3b37a00b8f None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:16.691020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ceb7f-a5e9-454b-9937-ad3b37a00b8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:16.691981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ceb7f-a5e9-454b-9937-ad3b37a00b8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:16.696617 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ceb7f-a5e9-454b-9937-ad3b37a00b8f None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:16.711075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 224/441] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:16.712473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06deae-3c90-40ca-a213-6299a0fc1c12 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:16.712762 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06deae-3c90-40ca-a213-6299a0fc1c12 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:16.713071 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06deae-3c90-40ca-a213-6299a0fc1c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:16.731752 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-ef06deae-3c90-40ca-a213-6299a0fc1c12 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:16.732990 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ef06deae-3c90-40ca-a213-6299a0fc1c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:16.800286 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a05b0-7cab-437f-a6ca-bbf4d09b060a None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:16.807036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 225/442] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:16.807926 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06deae-3c90-40ca-a213-6299a0fc1c12 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:16.844643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b426e3-5e08-42bf-846a-44deb144e738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:16.845508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b426e3-5e08-42bf-846a-44deb144e738 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:16.845791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b426e3-5e08-42bf-846a-44deb144e738 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:16.846069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b426e3-5e08-42bf-846a-44deb144e738 None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:16.847264 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b426e3-5e08-42bf-846a-44deb144e738 None admin] RBAC: Authorizing `identity:get_project(project_id=a89544d6d9d14e759c077d7175365dca)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:16.848786 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b426e3-5e08-42bf-846a-44deb144e738 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:16.850666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 218/443] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:32:16 2022] GET /identity/v3/projects/a89544d6d9d14e759c077d7175365dca => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:16.851613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b426e3-5e08-42bf-846a-44deb144e738 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:16.892504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:16.922103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06deae-3c90-40ca-a213-6299a0fc1c12 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:16.948013 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:16.966229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b426e3-5e08-42bf-846a-44deb144e738 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:16.966947 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59463f1a-d844-4e7c-b04c-6cd9c295170e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:16.967640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59463f1a-d844-4e7c-b04c-6cd9c295170e None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:16.967920 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59463f1a-d844-4e7c-b04c-6cd9c295170e None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:16.968195 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59463f1a-d844-4e7c-b04c-6cd9c295170e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:17.005839 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15846c-67c2-4362-a360-3936c94628ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:17.006668 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15846c-67c2-4362-a360-3936c94628ba None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:17.006949 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15846c-67c2-4362-a360-3936c94628ba None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:17.007230 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15846c-67c2-4362-a360-3936c94628ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:17.018750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15846c-67c2-4362-a360-3936c94628ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:17.026981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-6c15846c-67c2-4362-a360-3936c94628ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:32:17.026981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:17.026981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:17.026981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:17.026981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:17.026981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:17.026981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:17.026981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:17.026981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:17.026981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:17.026981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:17.026981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:17.026981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:17.026981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:17.026981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:17.026981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:17.028086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:17.028086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:17.028086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:17.028086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:17.028086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:17.028086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:17.028086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:17.028086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:17.028086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:17.028086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:17.028086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:17.028086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:17.028086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:17.028086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:17.028086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:17.028086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:17.028972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:17.028972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:17.028972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:17.028972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:17.028972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:17.028972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:17.028972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:17.028972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:17.028972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:17.028972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:17.028972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:32:17.028972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:17.028972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 219/444] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:32:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:17.029631 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15846c-67c2-4362-a360-3936c94628ba None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:17.051502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59463f1a-d844-4e7c-b04c-6cd9c295170e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:17.052959 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59463f1a-d844-4e7c-b04c-6cd9c295170e None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:17.133214 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:17.140878 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15846c-67c2-4362-a360-3936c94628ba None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:17.167203 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 226/445] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:17.167203 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59463f1a-d844-4e7c-b04c-6cd9c295170e None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:17.189315 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6ff75-f2f1-4563-9180-8bcc8f9c8d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:17.190275 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:17.190690 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6ff75-f2f1-4563-9180-8bcc8f9c8d91 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:17.190818 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6ff75-f2f1-4563-9180-8bcc8f9c8d91 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:17.191264 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6ff75-f2f1-4563-9180-8bcc8f9c8d91 None admin] PATH_INFO: `/v3/groups/b58cdca6be124c00b0deece5538c211c` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:17.193450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6ff75-f2f1-4563-9180-8bcc8f9c8d91 None admin] RBAC: Authorizing `identity:get_group(group_id=b58cdca6be124c00b0deece5538c211c)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:17.195200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6ff75-f2f1-4563-9180-8bcc8f9c8d91 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:17.196763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 220/446] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jun 22 11:32:17 2022] GET /identity/v3/groups/b58cdca6be124c00b0deece5538c211c => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:17.200785 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6ff75-f2f1-4563-9180-8bcc8f9c8d91 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:17.273730 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59463f1a-d844-4e7c-b04c-6cd9c295170e None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:17.314327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c088f20-e6ef-4a10-a5f3-e6bac7f4b334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:17.314827 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6ff75-f2f1-4563-9180-8bcc8f9c8d91 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:17.314992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c088f20-e6ef-4a10-a5f3-e6bac7f4b334 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:17.315272 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c088f20-e6ef-4a10-a5f3-e6bac7f4b334 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:17.315576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c088f20-e6ef-4a10-a5f3-e6bac7f4b334 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:17.316321 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c088f20-e6ef-4a10-a5f3-e6bac7f4b334 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:17.316741 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c088f20-e6ef-4a10-a5f3-e6bac7f4b334 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:17.318175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c088f20-e6ef-4a10-a5f3-e6bac7f4b334 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:17.328643 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 227/447] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:17.329722 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c088f20-e6ef-4a10-a5f3-e6bac7f4b334 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:17.354066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd640df-da47-41ed-bc25-f6f9628119a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:17.354874 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd640df-da47-41ed-bc25-f6f9628119a1 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:17.355157 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd640df-da47-41ed-bc25-f6f9628119a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:17.355468 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd640df-da47-41ed-bc25-f6f9628119a1 None admin] PATH_INFO: `/v3/projects/c77776cd83344a69a0a4fbb7efadd00f` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:17.356813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd640df-da47-41ed-bc25-f6f9628119a1 None admin] RBAC: Authorizing `identity:get_project(project_id=c77776cd83344a69a0a4fbb7efadd00f)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:17.358257 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd640df-da47-41ed-bc25-f6f9628119a1 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:17.360043 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 221/448] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:32:17 2022] GET /identity/v3/projects/c77776cd83344a69a0a4fbb7efadd00f => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:17.360818 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd640df-da47-41ed-bc25-f6f9628119a1 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:17.399030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:17.423737 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:17.447044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c088f20-e6ef-4a10-a5f3-e6bac7f4b334 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:17.477230 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd640df-da47-41ed-bc25-f6f9628119a1 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:17.491444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a3545-f7d3-440a-b91c-8d20fbf3fe19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:17.492549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a3545-f7d3-440a-b91c-8d20fbf3fe19 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:17.493040 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a3545-f7d3-440a-b91c-8d20fbf3fe19 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:17.493475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a3545-f7d3-440a-b91c-8d20fbf3fe19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:17.494501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a3545-f7d3-440a-b91c-8d20fbf3fe19 None admin] RBAC: Adding query filter params (group.id=b58cdca6be124c00b0deece5538c211c, role.id=f2336919c5104f6a8210e99994f4d455, scope.project.id=a89544d6d9d14e759c077d7175365dca) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:17.494825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a3545-f7d3-440a-b91c-8d20fbf3fe19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:17.496614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a3545-f7d3-440a-b91c-8d20fbf3fe19 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:17.502972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 228/449] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jun 22 11:32:17 2022] GET /identity/v3/role_assignments?group.id=b58cdca6be124c00b0deece5538c211c&scope.project.id=a89544d6d9d14e759c077d7175365dca&role.id=f2336919c5104f6a8210e99994f4d455 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:17.503747 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a3545-f7d3-440a-b91c-8d20fbf3fe19 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:17.518661 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7bc2d3-0e35-4525-893b-ccb2241cdd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:17.519516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bc2d3-0e35-4525-893b-ccb2241cdd1d None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:17.519937 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bc2d3-0e35-4525-893b-ccb2241cdd1d None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:17.520286 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bc2d3-0e35-4525-893b-ccb2241cdd1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:17.620438 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bc2d3-0e35-4525-893b-ccb2241cdd1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:17.623032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bc2d3-0e35-4525-893b-ccb2241cdd1d None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:17.649395 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a3545-f7d3-440a-b91c-8d20fbf3fe19 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:17.675215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:17.702787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8e454-feda-4f09-8c30-9917d21a53a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:17.704652 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8e454-feda-4f09-8c30-9917d21a53a1 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:17.704933 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8e454-feda-4f09-8c30-9917d21a53a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:17.705206 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8e454-feda-4f09-8c30-9917d21a53a1 None admin] PATH_INFO: `/v3/groups/8521ee997e8847b6bbeb83507f355e93` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:17.706621 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8e454-feda-4f09-8c30-9917d21a53a1 None admin] RBAC: Authorizing `identity:get_group(group_id=8521ee997e8847b6bbeb83507f355e93)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:17.711098 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8e454-feda-4f09-8c30-9917d21a53a1 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:17.712931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 229/450] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jun 22 11:32:17 2022] GET /identity/v3/groups/8521ee997e8847b6bbeb83507f355e93 => generated 229 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:17.718027 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8e454-feda-4f09-8c30-9917d21a53a1 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:17.742304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:17.785117 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 222/451] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:17.792102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7bc2d3-0e35-4525-893b-ccb2241cdd1d None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:17.885380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8e454-feda-4f09-8c30-9917d21a53a1 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:17.932079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68740bda-e00e-46af-bc53-f495beace1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:17.933719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68740bda-e00e-46af-bc53-f495beace1c0 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:17.934180 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68740bda-e00e-46af-bc53-f495beace1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:17.934599 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68740bda-e00e-46af-bc53-f495beace1c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:17.935683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68740bda-e00e-46af-bc53-f495beace1c0 None admin] RBAC: Adding query filter params (group.id=8521ee997e8847b6bbeb83507f355e93, role.id=96c584428d924fdba9671949f760c406, scope.project.id=c77776cd83344a69a0a4fbb7efadd00f) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:17.936270 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68740bda-e00e-46af-bc53-f495beace1c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:17.938478 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68740bda-e00e-46af-bc53-f495beace1c0 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:17.946708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 230/452] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jun 22 11:32:17 2022] GET /identity/v3/role_assignments?group.id=8521ee997e8847b6bbeb83507f355e93&scope.project.id=c77776cd83344a69a0a4fbb7efadd00f&role.id=96c584428d924fdba9671949f760c406 => generated 611 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:17.947292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7bc2d3-0e35-4525-893b-ccb2241cdd1d None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:17.987582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:17.994291 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad3a2e-d0d9-4707-b969-b4bc694ed599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:17.995231 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad3a2e-d0d9-4707-b969-b4bc694ed599 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:17.995632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad3a2e-d0d9-4707-b969-b4bc694ed599 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:17.995917 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad3a2e-d0d9-4707-b969-b4bc694ed599 None admin] PATH_INFO: `/v3/projects/a89544d6d9d14e759c077d7175365dca` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:17.997229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad3a2e-d0d9-4707-b969-b4bc694ed599 None admin] RBAC: Authorizing `identity:get_project(project_id=a89544d6d9d14e759c077d7175365dca)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:17.998729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad3a2e-d0d9-4707-b969-b4bc694ed599 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:18.003562 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 223/453] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:32:17 2022] GET /identity/v3/projects/a89544d6d9d14e759c077d7175365dca => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:18.007057 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68740bda-e00e-46af-bc53-f495beace1c0 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:18.108031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:18.139125 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68740bda-e00e-46af-bc53-f495beace1c0 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:18.182972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58160903-3d6f-4c61-9edb-241c25d3af74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:18.183874 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58160903-3d6f-4c61-9edb-241c25d3af74 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:18.184165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58160903-3d6f-4c61-9edb-241c25d3af74 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:18.184441 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58160903-3d6f-4c61-9edb-241c25d3af74 None admin] PATH_INFO: `/v3/groups/b58cdca6be124c00b0deece5538c211c` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:18.185688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58160903-3d6f-4c61-9edb-241c25d3af74 None admin] RBAC: Authorizing `identity:get_group(group_id=b58cdca6be124c00b0deece5538c211c)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:18.187125 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58160903-3d6f-4c61-9edb-241c25d3af74 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:18.188817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 231/454] 10.222.0.36 () {56 vars in 1180 bytes} [Wed Jun 22 11:32:18 2022] GET /identity/v3/groups/b58cdca6be124c00b0deece5538c211c => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:18.194833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad3a2e-d0d9-4707-b969-b4bc694ed599 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:18.296108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad3a2e-d0d9-4707-b969-b4bc694ed599 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:18.330299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa0ad6-2372-4d67-93bb-1059b21009f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:18.330995 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa0ad6-2372-4d67-93bb-1059b21009f3 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:18.331383 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa0ad6-2372-4d67-93bb-1059b21009f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:18.331712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa0ad6-2372-4d67-93bb-1059b21009f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:18.333093 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa0ad6-2372-4d67-93bb-1059b21009f3 None admin] RBAC: Adding query filter params (group.id=b58cdca6be124c00b0deece5538c211c, role.id=b4be7021e4104740b8b0b3b40ecd3d59, scope.project.id=a89544d6d9d14e759c077d7175365dca) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:18.333402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa0ad6-2372-4d67-93bb-1059b21009f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:18.334843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa0ad6-2372-4d67-93bb-1059b21009f3 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:18.341111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 224/455] 10.222.0.36 () {56 vars in 1376 bytes} [Wed Jun 22 11:32:18 2022] GET /identity/v3/role_assignments?group.id=b58cdca6be124c00b0deece5538c211c&scope.project.id=a89544d6d9d14e759c077d7175365dca&role.id=b4be7021e4104740b8b0b3b40ecd3d59 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:21.431207 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccd85e-b3ab-41be-a93f-4c7b28210dae None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:21.431207 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccd85e-b3ab-41be-a93f-4c7b28210dae None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:21.431207 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccd85e-b3ab-41be-a93f-4c7b28210dae None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:21.432300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 232/456] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:21.437999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a17b0-bc6c-4c22-ae85-832ffa3a1d8b None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:21.438393 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a17b0-bc6c-4c22-ae85-832ffa3a1d8b None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:21.438591 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a17b0-bc6c-4c22-ae85-832ffa3a1d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:21.468409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-eb0a17b0-bc6c-4c22-ae85-832ffa3a1d8b None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:21.469609 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-eb0a17b0-bc6c-4c22-ae85-832ffa3a1d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:21.628048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:21.658935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 225/457] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:21.696723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ccd85e-b3ab-41be-a93f-4c7b28210dae None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:21.934390 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ccd85e-b3ab-41be-a93f-4c7b28210dae None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:21.954338 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43e757-b799-41fc-8680-6a4d38c1bf06 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:21.954756 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43e757-b799-41fc-8680-6a4d38c1bf06 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:21.955574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43e757-b799-41fc-8680-6a4d38c1bf06 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:21.958949 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 226/458] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:21.975840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b44bb-d391-42bb-b300-dcf7ca62eecf None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:21.976655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b44bb-d391-42bb-b300-dcf7ca62eecf None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:21.977288 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b44bb-d391-42bb-b300-dcf7ca62eecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:22.044737 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-2a7b44bb-d391-42bb-b300-dcf7ca62eecf None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:22.046901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-2a7b44bb-d391-42bb-b300-dcf7ca62eecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:22.058355 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2615d7-cda3-4569-bf3a-50b97dd4ff89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:22.060308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2615d7-cda3-4569-bf3a-50b97dd4ff89 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:22.063944 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2615d7-cda3-4569-bf3a-50b97dd4ff89 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:22.068113 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2615d7-cda3-4569-bf3a-50b97dd4ff89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:22.115019 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2615d7-cda3-4569-bf3a-50b97dd4ff89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:22.116631 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2615d7-cda3-4569-bf3a-50b97dd4ff89 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:22.135311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:22.176134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-1a2615d7-cda3-4569-bf3a-50b97dd4ff89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:22.176134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:22.176134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:22.176134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:22.176134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:22.176134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:22.176134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:22.176134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:22.176134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:22.176134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:22.176134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:22.176134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:22.176134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:22.176134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:22.176134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:22.177053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:22.177053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:22.177053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:22.177053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:22.177053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:22.177053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:22.177053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:22.177053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:22.177053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:22.177053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:22.177053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:22.177053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:22.177053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:22.177053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:22.177053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:22.177053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:22.177053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:22.177914 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:22.177914 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:22.177914 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:22.177914 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:22.179803 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 233/459] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 484 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:22.185734 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2615d7-cda3-4569-bf3a-50b97dd4ff89 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:22.327422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 227/460] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:22.377918 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b44bb-d391-42bb-b300-dcf7ca62eecf None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:22.546050 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b44bb-d391-42bb-b300-dcf7ca62eecf None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:22.575303 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2615d7-cda3-4569-bf3a-50b97dd4ff89 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:22.581195 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:22.595211 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673fb239-a5ef-449a-90c3-b13a4640d474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:22.596840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fb239-a5ef-449a-90c3-b13a4640d474 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:22.597321 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fb239-a5ef-449a-90c3-b13a4640d474 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:22.597827 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fb239-a5ef-449a-90c3-b13a4640d474 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:22.610430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673fb239-a5ef-449a-90c3-b13a4640d474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:22.612838 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673fb239-a5ef-449a-90c3-b13a4640d474 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:22.640736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-673fb239-a5ef-449a-90c3-b13a4640d474 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:22.640736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:22.640736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:22.640736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:22.640736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:22.640736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:22.640736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:22.640736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:22.640736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:22.640736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:22.640736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:22.640736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:22.640736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:22.640736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:22.640736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:22.641619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:22.641619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:22.641619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:22.641619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:22.641619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:22.641619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:22.641619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:22.641619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:22.641619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:22.641619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:22.641619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:22.641619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:22.641619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:22.641619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:22.641619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:22.641619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:22.641619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:22.642454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:22.642454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:22.642454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:22.642454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:22.643967 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51bb071-c4b1-4360-a939-6747b4b0e02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:22.644692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 228/461] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:22.644970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bb071-c4b1-4360-a939-6747b4b0e02e None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:22.645281 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bb071-c4b1-4360-a939-6747b4b0e02e None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:22.645817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51bb071-c4b1-4360-a939-6747b4b0e02e None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:22.646812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f5ec8-b5ec-452e-a39b-cc462eefe393 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:22.646935 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51bb071-c4b1-4360-a939-6747b4b0e02e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:22.647461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f5ec8-b5ec-452e-a39b-cc462eefe393 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:22.647560 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51bb071-c4b1-4360-a939-6747b4b0e02e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:22.647991 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f5ec8-b5ec-452e-a39b-cc462eefe393 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:22.653210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51bb071-c4b1-4360-a939-6747b4b0e02e None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:22.653595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 229/462] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:22.655004 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d4dab-004f-4806-92f2-f10a7952239f None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:22.655473 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d4dab-004f-4806-92f2-f10a7952239f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:22.655905 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d4dab-004f-4806-92f2-f10a7952239f None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:22.657568 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 230/463] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:22.658395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d4dab-004f-4806-92f2-f10a7952239f None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:22.671517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 234/464] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 486 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:22.672893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1aeb8-2b27-4573-89c9-72c5ba68a231 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:22.673186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1aeb8-2b27-4573-89c9-72c5ba68a231 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:22.673465 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1aeb8-2b27-4573-89c9-72c5ba68a231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:22.692601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-1cc1aeb8-2b27-4573-89c9-72c5ba68a231 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:22.694002 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-1cc1aeb8-2b27-4573-89c9-72c5ba68a231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:22.758873 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:22.791168 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 235/465] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:22.792912 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca9907-fd2c-4422-93bb-9f8e9b7c86ac None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:22.793405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca9907-fd2c-4422-93bb-9f8e9b7c86ac None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:22.803374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca9907-fd2c-4422-93bb-9f8e9b7c86ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:22.818114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d4dab-004f-4806-92f2-f10a7952239f None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:22.829324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-9fca9907-fd2c-4422-93bb-9f8e9b7c86ac None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:22.830877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-9fca9907-fd2c-4422-93bb-9f8e9b7c86ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:22.865754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfeda18-2ba5-4306-abda-9fbc2c998627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:22.866814 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfeda18-2ba5-4306-abda-9fbc2c998627 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:22.867292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfeda18-2ba5-4306-abda-9fbc2c998627 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:22.867778 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfeda18-2ba5-4306-abda-9fbc2c998627 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:22.868678 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfeda18-2ba5-4306-abda-9fbc2c998627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:22.869379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfeda18-2ba5-4306-abda-9fbc2c998627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:22.871899 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfeda18-2ba5-4306-abda-9fbc2c998627 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:22.892864 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 231/466] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:22.894246 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfeda18-2ba5-4306-abda-9fbc2c998627 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:22.908352 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 236/467] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:22.909639 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca9907-fd2c-4422-93bb-9f8e9b7c86ac None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:22.946800 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:23.032374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca9907-fd2c-4422-93bb-9f8e9b7c86ac None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:23.054538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:23.057120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfeda18-2ba5-4306-abda-9fbc2c998627 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:23.077487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c70170-4912-4194-80cf-1f4f73a60f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:23.078884 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c70170-4912-4194-80cf-1f4f73a60f08 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:23.079167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c70170-4912-4194-80cf-1f4f73a60f08 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:23.079470 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c70170-4912-4194-80cf-1f4f73a60f08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:23.090145 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c70170-4912-4194-80cf-1f4f73a60f08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:23.091629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c70170-4912-4194-80cf-1f4f73a60f08 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:23.101229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76578942-1d8f-4841-9b79-bc279f8ee4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:23.101907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76578942-1d8f-4841-9b79-bc279f8ee4c2 None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:23.102360 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76578942-1d8f-4841-9b79-bc279f8ee4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:23.102704 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76578942-1d8f-4841-9b79-bc279f8ee4c2 None admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:23.103523 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76578942-1d8f-4841-9b79-bc279f8ee4c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:23.105008 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76578942-1d8f-4841-9b79-bc279f8ee4c2 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:23.111169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-31c70170-4912-4194-80cf-1f4f73a60f08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:23.111169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:23.111169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:23.111169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:23.111169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:23.111169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:23.111169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:23.111169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:23.111169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:23.111169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:23.111169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:23.111169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:23.111169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:23.111169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:23.111169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:23.112159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:23.112159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:23.112159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:23.112159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:23.112159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:23.112159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:23.112159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:23.112159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:23.112159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:23.112159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:23.112159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:23.112159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:23.112159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:23.112159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:23.112159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:23.112159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:23.112159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:23.112991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:23.112991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:23.112991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:23.112991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:23.112991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 237/468] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:23.113837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c70170-4912-4194-80cf-1f4f73a60f08 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:23.132970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 232/469] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jun 22 11:32:22 2022] POST /identity/v3/users => generated 246 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:32:23.134227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76578942-1d8f-4841-9b79-bc279f8ee4c2 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:23.260612 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c70170-4912-4194-80cf-1f4f73a60f08 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:23.273465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:23.286260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76578942-1d8f-4841-9b79-bc279f8ee4c2 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:23.311234 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfca26-3365-4bc3-95a2-51566799ef3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:23.312104 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfca26-3365-4bc3-95a2-51566799ef3f None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:23.312646 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfca26-3365-4bc3-95a2-51566799ef3f None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:23.313742 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfca26-3365-4bc3-95a2-51566799ef3f None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:23.314722 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edfca26-3365-4bc3-95a2-51566799ef3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:23.317475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edfca26-3365-4bc3-95a2-51566799ef3f None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:23.330770 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4364ab-585a-4d30-bbc7-722e461eaa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:23.330949 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4364ab-585a-4d30-bbc7-722e461eaa25 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:23.331163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4364ab-585a-4d30-bbc7-722e461eaa25 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:23.331628 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4364ab-585a-4d30-bbc7-722e461eaa25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:23.348621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4364ab-585a-4d30-bbc7-722e461eaa25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:23.350185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4364ab-585a-4d30-bbc7-722e461eaa25 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:23.350521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 238/470] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jun 22 11:32:23 2022] POST /identity/v3/users => generated 248 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:32:23.361166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97856df1-50c7-4cba-b8a8-150d313223d6 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:23.361655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97856df1-50c7-4cba-b8a8-150d313223d6 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:23.362076 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97856df1-50c7-4cba-b8a8-150d313223d6 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:23.364042 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 239/471] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:23 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:23.364991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97856df1-50c7-4cba-b8a8-150d313223d6 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:23.387935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-1c4364ab-585a-4d30-bbc7-722e461eaa25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:23.387935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:23.387935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:23.387935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:23.387935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:23.387935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:23.387935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:23.387935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:23.387935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:23.387935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:23.387935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:23.387935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:23.387935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:23.387935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:23.387935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:23.388844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:23.388844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:23.388844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:23.388844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:23.388844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:23.388844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:23.388844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:23.388844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:23.388844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:23.388844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:23.388844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:23.388844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:23.388844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:23.388844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:23.388844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:23.388844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:23.388844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:23.389698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:23.389698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:23.389698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:23.389698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:23.390584 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 233/472] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:23.391693 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a83494-3a29-478e-8707-f349a1430e63 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:23.391984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a83494-3a29-478e-8707-f349a1430e63 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:23.392480 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a83494-3a29-478e-8707-f349a1430e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:23.409379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:23.419384 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-65a83494-3a29-478e-8707-f349a1430e63 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:23.419384 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-65a83494-3a29-478e-8707-f349a1430e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:23.582918 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 234/473] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:23.583761 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a83494-3a29-478e-8707-f349a1430e63 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:23.592873 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97856df1-50c7-4cba-b8a8-150d313223d6 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:23.666858 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c3a67-d6c2-4525-b311-8698ee7211fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:23.667906 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c3a67-d6c2-4525-b311-8698ee7211fc None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:23.668524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c3a67-d6c2-4525-b311-8698ee7211fc None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:23.668999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c3a67-d6c2-4525-b311-8698ee7211fc None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:23.669942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c3a67-d6c2-4525-b311-8698ee7211fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:23.670443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c3a67-d6c2-4525-b311-8698ee7211fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:23.673086 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c3a67-d6c2-4525-b311-8698ee7211fc None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:23.688618 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 240/474] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:23.689713 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c3a67-d6c2-4525-b311-8698ee7211fc None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:23.747694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:23.851022 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a83494-3a29-478e-8707-f349a1430e63 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:23.950580 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c58a6-deaa-4c03-bce4-513e807e44d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:23.952680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c58a6-deaa-4c03-bce4-513e807e44d6 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:23.953897 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c58a6-deaa-4c03-bce4-513e807e44d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:23.954696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c58a6-deaa-4c03-bce4-513e807e44d6 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:23.957299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c58a6-deaa-4c03-bce4-513e807e44d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:23.958131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c58a6-deaa-4c03-bce4-513e807e44d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:23.967378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c58a6-deaa-4c03-bce4-513e807e44d6 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:24.004877 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 235/475] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:24.006799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c58a6-deaa-4c03-bce4-513e807e44d6 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:24.075786 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:24.130382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c3a67-d6c2-4525-b311-8698ee7211fc None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:24.169869 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7577973-bd93-4de7-b3a9-1b2d5d929437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:24.170535 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7577973-bd93-4de7-b3a9-1b2d5d929437 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:24.170816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7577973-bd93-4de7-b3a9-1b2d5d929437 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:24.171091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7577973-bd93-4de7-b3a9-1b2d5d929437 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:24.171870 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7577973-bd93-4de7-b3a9-1b2d5d929437 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:24.173226 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7577973-bd93-4de7-b3a9-1b2d5d929437 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:24.180123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.sql.core [None req-d7577973-bd93-4de7-b3a9-1b2d5d929437 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 22 11:32:24.180123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 22 11:32:24.180123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [parameters: {'id': '25b4242ffe004db5b87fccc7614da2f9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 22 11:32:24.180123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86768) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 22 11:32:24.202087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c58a6-deaa-4c03-bce4-513e807e44d6 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:24.206334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-d7577973-bd93-4de7-b3a9-1b2d5d929437 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 22 11:32:24.206334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:24.206334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 22 11:32:24.206334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 22 11:32:24.206334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 22 11:32:24.206334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 22 11:32:24.206334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 22 11:32:24.206334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application result = self._query(query) Jun 22 11:32:24.206334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 22 11:32:24.206334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application conn.query(q) Jun 22 11:32:24.206334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 22 11:32:24.206334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 22 11:32:24.206334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 22 11:32:24.206334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application result.read() Jun 22 11:32:24.206334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 22 11:32:24.207274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 22 11:32:24.207274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 22 11:32:24.207274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 22 11:32:24.207274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 22 11:32:24.207274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 22 11:32:24.207274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 22 11:32:24.207274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 22 11:32:24.207274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 22 11:32:24.207274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:24.207274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 22 11:32:24.207274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:24.207274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:24.207274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 22 11:32:24.207274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 22 11:32:24.207274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 22 11:32:24.207274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return ref.to_dict() Jun 22 11:32:24.207274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 22 11:32:24.207274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application next(self.gen) Jun 22 11:32:24.208548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 22 11:32:24.208548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application yield resource Jun 22 11:32:24.208548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 22 11:32:24.208548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application next(self.gen) Jun 22 11:32:24.208548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 22 11:32:24.208548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self.session.rollback() Jun 22 11:32:24.208548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 22 11:32:24.208548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self.force_reraise() Jun 22 11:32:24.208548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 22 11:32:24.208548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise self.value Jun 22 11:32:24.208548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 22 11:32:24.208548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 22 11:32:24.208548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 22 11:32:24.208548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application session.commit() Jun 22 11:32:24.208548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 22 11:32:24.208548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 22 11:32:24.208548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 22 11:32:24.209930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._prepare_impl() Jun 22 11:32:24.209930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 22 11:32:24.209930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self.session.flush() Jun 22 11:32:24.209930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 22 11:32:24.209930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._flush(objects) Jun 22 11:32:24.209930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 22 11:32:24.209930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 22 11:32:24.209930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 22 11:32:24.209930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application compat.raise_( Jun 22 11:32:24.209930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 22 11:32:24.209930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception Jun 22 11:32:24.209930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 22 11:32:24.209930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application flush_context.execute() Jun 22 11:32:24.209930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 22 11:32:24.209930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rec.execute(self) Jun 22 11:32:24.209930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 22 11:32:24.209930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 22 11:32:24.211466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 22 11:32:24.211466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 22 11:32:24.211466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 22 11:32:24.211466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 22 11:32:24.211466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 22 11:32:24.211466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 22 11:32:24.211466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 22 11:32:24.211466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 22 11:32:24.211466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 22 11:32:24.211466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 22 11:32:24.211466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 22 11:32:24.211466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 22 11:32:24.211466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 22 11:32:24.211466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 22 11:32:24.211466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 22 11:32:24.211466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception Jun 22 11:32:24.220084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 22 11:32:24.220084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 22 11:32:24.220084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 22 11:32:24.220084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 22 11:32:24.220084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 22 11:32:24.220084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application result = self._query(query) Jun 22 11:32:24.220084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 22 11:32:24.220084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application conn.query(q) Jun 22 11:32:24.220084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 22 11:32:24.220084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 22 11:32:24.220084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 22 11:32:24.220084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application result.read() Jun 22 11:32:24.220084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 22 11:32:24.220084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 22 11:32:24.220084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 22 11:32:24.220084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 22 11:32:24.221778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 22 11:32:24.221778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 22 11:32:24.221778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 22 11:32:24.221778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 22 11:32:24.221778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 22 11:32:24.221778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 22 11:32:24.221778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [parameters: {'id': '25b4242ffe004db5b87fccc7614da2f9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 22 11:32:24.221778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 22 11:32:24.221778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:24.221778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:32:24.221778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:24.221778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:24.221778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:24.221778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:24.221778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:24.221778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:24.223562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:24.223562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:24.223562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:24.223562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:24.223562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:24.223562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:24.223562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 22 11:32:24.223562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 22 11:32:24.223562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:24.223562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:24.223562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 22 11:32:24.223562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 22 11:32:24.223562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 22 11:32:24.223562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 22 11:32:24.223562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 22 11:32:24.223562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:24.224593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 241/476] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jun 22 11:32:23 2022] POST /identity/v3/roles => generated 144 bytes in 519 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:24.224593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7577973-bd93-4de7-b3a9-1b2d5d929437 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:24.245785 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98caec1c-6712-41a1-aaea-829321315a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:24.246599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caec1c-6712-41a1-aaea-829321315a4d None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:24.247164 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caec1c-6712-41a1-aaea-829321315a4d None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:24.247337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98caec1c-6712-41a1-aaea-829321315a4d None admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:24.248171 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98caec1c-6712-41a1-aaea-829321315a4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:24.249742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98caec1c-6712-41a1-aaea-829321315a4d None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:24.257400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:24.279337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 236/477] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jun 22 11:32:24 2022] POST /identity/v3/users => generated 249 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:32:24.280213 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98caec1c-6712-41a1-aaea-829321315a4d None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:24.334975 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7577973-bd93-4de7-b3a9-1b2d5d929437 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:24.406993 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927a535-2e5e-4afd-841e-17054c49c9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:24.407740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927a535-2e5e-4afd-841e-17054c49c9ee None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:24.408025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927a535-2e5e-4afd-841e-17054c49c9ee None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:24.408307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927a535-2e5e-4afd-841e-17054c49c9ee None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:24.409033 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927a535-2e5e-4afd-841e-17054c49c9ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:24.410629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927a535-2e5e-4afd-841e-17054c49c9ee None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:24.419265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98caec1c-6712-41a1-aaea-829321315a4d None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:24.455970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 242/478] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jun 22 11:32:24 2022] POST /identity/v3/users => generated 248 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:32:24.469502 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd1251-c7ce-4cc5-a37e-dfa89b5e233e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:24.470451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd1251-c7ce-4cc5-a37e-dfa89b5e233e None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:24.470734 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd1251-c7ce-4cc5-a37e-dfa89b5e233e None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:24.471007 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd1251-c7ce-4cc5-a37e-dfa89b5e233e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:24.483630 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd1251-c7ce-4cc5-a37e-dfa89b5e233e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:24.492763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-becd1251-c7ce-4cc5-a37e-dfa89b5e233e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:32:24.492763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:24.492763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:24.492763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:24.492763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:24.492763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:24.492763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:24.492763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:24.492763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:24.492763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:24.492763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:24.492763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:24.492763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:24.492763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:24.492763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:24.492763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:24.495021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:24.495021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:24.495021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:24.495021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:24.495021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:24.495021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:24.495021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:24.495021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:24.495021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:24.495021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:24.495021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:24.495021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:24.495021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:24.495021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:24.495021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:24.495021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:24.496981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:24.496981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:24.496981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:24.496981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:24.496981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:24.496981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:24.496981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:24.496981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:24.496981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:24.496981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:24.496981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 22 11:32:24.496981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:24.496981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 237/479] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:32:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:24.503377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927a535-2e5e-4afd-841e-17054c49c9ee None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:24.510673 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:24.624160 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927a535-2e5e-4afd-841e-17054c49c9ee None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:24.662729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cfec85-4d67-413d-ad5a-a5c0565c9f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:24.663797 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfec85-4d67-413d-ad5a-a5c0565c9f63 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:24.664304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfec85-4d67-413d-ad5a-a5c0565c9f63 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:24.664835 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfec85-4d67-413d-ad5a-a5c0565c9f63 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:24.665821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cfec85-4d67-413d-ad5a-a5c0565c9f63 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:24.666343 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cfec85-4d67-413d-ad5a-a5c0565c9f63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:24.667998 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cfec85-4d67-413d-ad5a-a5c0565c9f63 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:24.679411 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 243/480] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:25.527884 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca142660-71e2-410d-89f6-a83e39064237 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:25.528382 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca142660-71e2-410d-89f6-a83e39064237 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:25.528454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca142660-71e2-410d-89f6-a83e39064237 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:25.529980 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 238/481] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:25.536281 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dead88-2d36-4c17-a167-6e1f232ba5dd None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:25.536577 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dead88-2d36-4c17-a167-6e1f232ba5dd None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:25.536858 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dead88-2d36-4c17-a167-6e1f232ba5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:25.555950 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-e4dead88-2d36-4c17-a167-6e1f232ba5dd None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:25.557588 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e4dead88-2d36-4c17-a167-6e1f232ba5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:25.671736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:25.685144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 244/482] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:25.728543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca142660-71e2-410d-89f6-a83e39064237 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:25.844646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:25.919452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76a9e9-c102-4433-a54f-91978a2fc871 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:25.919452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76a9e9-c102-4433-a54f-91978a2fc871 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:25.919452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba76a9e9-c102-4433-a54f-91978a2fc871 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:25.920770 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 245/483] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:25.934762 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db97c5e-0284-40cf-9547-5e70a1f1a549 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:25.936501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db97c5e-0284-40cf-9547-5e70a1f1a549 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:25.937021 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db97c5e-0284-40cf-9547-5e70a1f1a549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:25.965506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-4db97c5e-0284-40cf-9547-5e70a1f1a549 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:25.967028 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-4db97c5e-0284-40cf-9547-5e70a1f1a549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:25.979310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca142660-71e2-410d-89f6-a83e39064237 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:26.036735 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a3275-e77c-4fcd-ae27-40718be1d48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:26.037876 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a3275-e77c-4fcd-ae27-40718be1d48a None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:26.038380 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a3275-e77c-4fcd-ae27-40718be1d48a None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:26.038864 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a3275-e77c-4fcd-ae27-40718be1d48a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:26.058414 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a3275-e77c-4fcd-ae27-40718be1d48a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:26.071558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-055a3275-e77c-4fcd-ae27-40718be1d48a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:26.071558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:26.071558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:26.071558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:26.071558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:26.071558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:26.071558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:26.071558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:26.071558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:26.071558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:26.071558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:26.071558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:26.071558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:26.071558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:26.071558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:26.071558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:26.072670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:26.072670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:26.072670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:26.072670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:26.072670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:26.072670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:26.072670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:26.072670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:26.072670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:26.072670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:26.072670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:26.072670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:26.072670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:26.072670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:26.072670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:26.072670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:26.073627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:26.073627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:26.073627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:26.073627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:26.073627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:26.073627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:26.073627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:26.073627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:26.073627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:26.073627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:26.073627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:26.073627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:26.075917 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 239/484] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:26.078063 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 246/485] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:26.081718 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a3275-e77c-4fcd-ae27-40718be1d48a None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:26.115058 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db97c5e-0284-40cf-9547-5e70a1f1a549 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:26.254672 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a3275-e77c-4fcd-ae27-40718be1d48a None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:26.266790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:26.301338 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6e662-a555-48a2-be0d-415acd703c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:26.302406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6e662-a555-48a2-be0d-415acd703c40 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:26.302919 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6e662-a555-48a2-be0d-415acd703c40 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:26.303424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6e662-a555-48a2-be0d-415acd703c40 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:26.304381 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6e662-a555-48a2-be0d-415acd703c40 None admin] RBAC: Adding query filter params (name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:26.304939 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6e662-a555-48a2-be0d-415acd703c40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:26.306609 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6e662-a555-48a2-be0d-415acd703c40 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:26.308182 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db97c5e-0284-40cf-9547-5e70a1f1a549 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:26.317458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 240/486] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:32:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:26.323476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6e662-a555-48a2-be0d-415acd703c40 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:26.370253 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:26.385039 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc3c53-9361-4efc-86f5-6cc084c9b817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:26.386073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc3c53-9361-4efc-86f5-6cc084c9b817 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:26.386523 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc3c53-9361-4efc-86f5-6cc084c9b817 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:26.386935 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc3c53-9361-4efc-86f5-6cc084c9b817 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:26.416373 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc3c53-9361-4efc-86f5-6cc084c9b817 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:26.432601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-ddbc3c53-9361-4efc-86f5-6cc084c9b817 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:26.432601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:26.432601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:26.432601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:26.432601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:26.432601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:26.432601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:26.432601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:26.432601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:26.432601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:26.432601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:26.432601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:26.432601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:26.432601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:26.432601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:26.432601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:26.433583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:26.433583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:26.433583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:26.433583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:26.433583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:26.433583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:26.433583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:26.433583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:26.433583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:26.433583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:26.433583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:26.433583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:26.433583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:26.433583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:26.433583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:26.433583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:26.434374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:26.434374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:26.434374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:26.434374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:26.434374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:26.434374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:26.434374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:26.434374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:26.434374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:26.434374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:26.434374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:26.434374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:26.434374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 247/487] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:32:26.439550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6e662-a555-48a2-be0d-415acd703c40 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:26.444436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc3c53-9361-4efc-86f5-6cc084c9b817 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:26.478821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc597fb-e867-4ed1-8820-c05cfaedfa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:26.479977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc597fb-e867-4ed1-8820-c05cfaedfa57 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:26.480433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc597fb-e867-4ed1-8820-c05cfaedfa57 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:26.480860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc597fb-e867-4ed1-8820-c05cfaedfa57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:26.494722 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc597fb-e867-4ed1-8820-c05cfaedfa57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:26.496591 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc597fb-e867-4ed1-8820-c05cfaedfa57 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:26.515885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-9dc597fb-e867-4ed1-8820-c05cfaedfa57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:26.515885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:26.515885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:26.515885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:26.515885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:26.515885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:26.515885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:26.515885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:26.515885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:26.515885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:26.515885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:26.515885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:26.515885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:26.515885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:26.515885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:26.516958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:26.516958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:26.516958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:26.516958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:26.516958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:26.516958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:26.516958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:26.516958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:26.516958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:26.516958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:26.516958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:26.516958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:26.516958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:26.516958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:26.516958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:26.516958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:26.516958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:26.517934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:26.517934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:26.517934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:26.517934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:26.520504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 241/488] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:32:26.527748 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc597fb-e867-4ed1-8820-c05cfaedfa57 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:26.564884 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:26.673524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc597fb-e867-4ed1-8820-c05cfaedfa57 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:26.711517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc3c53-9361-4efc-86f5-6cc084c9b817 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:26.748118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83677d2-d859-4aac-99eb-db9c1a16d54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:26.748800 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83677d2-d859-4aac-99eb-db9c1a16d54c None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:26.749082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83677d2-d859-4aac-99eb-db9c1a16d54c None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:26.749357 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83677d2-d859-4aac-99eb-db9c1a16d54c None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:26.750087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83677d2-d859-4aac-99eb-db9c1a16d54c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:26.750390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83677d2-d859-4aac-99eb-db9c1a16d54c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:26.751865 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83677d2-d859-4aac-99eb-db9c1a16d54c None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:26.778316 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 242/489] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:26.781564 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83677d2-d859-4aac-99eb-db9c1a16d54c None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:26.800166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d34dd-4321-4875-bb73-2b4af096b67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:26.800958 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d34dd-4321-4875-bb73-2b4af096b67b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:26.801247 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d34dd-4321-4875-bb73-2b4af096b67b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:26.801523 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d34dd-4321-4875-bb73-2b4af096b67b None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:26.802318 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d34dd-4321-4875-bb73-2b4af096b67b None admin] RBAC: Adding query filter params (name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:26.802946 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d34dd-4321-4875-bb73-2b4af096b67b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:26.804705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d34dd-4321-4875-bb73-2b4af096b67b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:26.822844 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 248/490] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:32:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:26.828016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d34dd-4321-4875-bb73-2b4af096b67b None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:26.887074 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:26.909212 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83677d2-d859-4aac-99eb-db9c1a16d54c None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:26.925034 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:26.966166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132ced8-41e7-401f-95bc-a5711e1088bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:26.967470 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132ced8-41e7-401f-95bc-a5711e1088bd None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:26.967855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132ced8-41e7-401f-95bc-a5711e1088bd None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:26.968227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132ced8-41e7-401f-95bc-a5711e1088bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:26.978449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132ced8-41e7-401f-95bc-a5711e1088bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:26.980840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132ced8-41e7-401f-95bc-a5711e1088bd None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:26.986283 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d34dd-4321-4875-bb73-2b4af096b67b None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:27.002378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-a132ced8-41e7-401f-95bc-a5711e1088bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 22 11:32:27.002378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:27.002378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:27.002378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:27.002378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:27.002378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:27.002378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:27.002378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:27.002378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:27.002378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:27.002378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:27.002378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:27.002378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:27.002378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:27.002378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:27.002378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:27.003504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:27.003504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:27.003504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:27.003504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:27.003504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:27.003504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:27.003504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:27.003504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:27.003504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:27.003504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:27.003504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:27.003504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:27.003504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:27.003504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:27.003504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:27.003504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:27.003504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:27.003504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:27.005039 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:27.005039 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:27.005039 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:27.005039 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:27.005039 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:27.005039 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:27.005039 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:27.005039 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:27.005039 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 22 11:32:27.005039 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:27.005562 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 243/491] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jun 22 11:32:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:32:27.006612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334c60c-fd8d-4653-b95b-2476f47d28ab None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:27.007043 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334c60c-fd8d-4653-b95b-2476f47d28ab None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:27.007476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334c60c-fd8d-4653-b95b-2476f47d28ab None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:27.009006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 244/492] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:27.014687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334c60c-fd8d-4653-b95b-2476f47d28ab None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:27.039676 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f6415-4941-4965-b001-e69d7de20311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:27.041154 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f6415-4941-4965-b001-e69d7de20311 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:27.041608 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f6415-4941-4965-b001-e69d7de20311 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:27.042021 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f6415-4941-4965-b001-e69d7de20311 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:27.059596 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f6415-4941-4965-b001-e69d7de20311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:27.060968 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f6415-4941-4965-b001-e69d7de20311 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:27.096134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-3d4f6415-4941-4965-b001-e69d7de20311 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:27.096134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:27.096134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:27.096134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:27.096134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:27.096134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:27.096134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:27.096134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:27.096134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:27.096134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:27.096134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:27.096134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:27.096134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:27.096134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:27.096134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:27.097025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:27.097025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:27.097025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:27.097025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:27.097025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:27.097025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:27.097025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:27.097025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:27.097025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:27.097025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:27.097025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:27.097025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:27.097025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:27.097025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:27.097025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:27.097025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:27.097025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:27.098627 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:27.098627 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:27.098627 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:27.098627 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:27.106231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 249/493] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:27.106231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3caa0e4-3e9f-4345-b344-9d27f7e3cc54 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:27.106231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3caa0e4-3e9f-4345-b344-9d27f7e3cc54 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:27.106231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3caa0e4-3e9f-4345-b344-9d27f7e3cc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:27.133211 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-b3caa0e4-3e9f-4345-b344-9d27f7e3cc54 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:27.135555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b3caa0e4-3e9f-4345-b344-9d27f7e3cc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:27.144559 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334c60c-fd8d-4653-b95b-2476f47d28ab None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:27.189916 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db54e85-58cf-4202-a564-4785048c9ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:27.190547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db54e85-58cf-4202-a564-4785048c9ed1 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:27.190843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db54e85-58cf-4202-a564-4785048c9ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:27.191114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db54e85-58cf-4202-a564-4785048c9ed1 None admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:27.192161 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db54e85-58cf-4202-a564-4785048c9ed1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:27.192471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db54e85-58cf-4202-a564-4785048c9ed1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:27.193997 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db54e85-58cf-4202-a564-4785048c9ed1 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:27.239051 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 250/495] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:27.239298 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 245/495] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jun 22 11:32:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:27.240220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db54e85-58cf-4202-a564-4785048c9ed1 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:27.240914 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3caa0e4-3e9f-4345-b344-9d27f7e3cc54 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:27.244244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:27.297499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:27.383989 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3caa0e4-3e9f-4345-b344-9d27f7e3cc54 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:27.388499 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db54e85-58cf-4202-a564-4785048c9ed1 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:27.431341 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb42019-bfe6-4797-b2e0-fbde106f5f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:27.431658 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f06303-f238-4464-8c42-b12754fb4328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:27.432029 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb42019-bfe6-4797-b2e0-fbde106f5f17 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:27.432310 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb42019-bfe6-4797-b2e0-fbde106f5f17 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:27.432586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb42019-bfe6-4797-b2e0-fbde106f5f17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:27.433108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f06303-f238-4464-8c42-b12754fb4328 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:27.433555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f06303-f238-4464-8c42-b12754fb4328 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:27.433843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f06303-f238-4464-8c42-b12754fb4328 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:27.434628 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f06303-f238-4464-8c42-b12754fb4328 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:27.434943 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f06303-f238-4464-8c42-b12754fb4328 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:27.436660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f06303-f238-4464-8c42-b12754fb4328 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:27.459569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 246/496] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:27.459569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f06303-f238-4464-8c42-b12754fb4328 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:27.511570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb42019-bfe6-4797-b2e0-fbde106f5f17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:27.513054 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb42019-bfe6-4797-b2e0-fbde106f5f17 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:27.544919 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:27.548368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:27.586499 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f06303-f238-4464-8c42-b12754fb4328 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:27.621761 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f45279-1234-420a-89a9-d9c55b645c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:27.622548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f45279-1234-420a-89a9-d9c55b645c98 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:27.622830 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f45279-1234-420a-89a9-d9c55b645c98 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:27.623107 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f45279-1234-420a-89a9-d9c55b645c98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:27.623736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 251/497] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:27.625133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c6c1e-ecad-4b13-8870-53c98e940de8 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:27.625421 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c6c1e-ecad-4b13-8870-53c98e940de8 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:27.625695 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c6c1e-ecad-4b13-8870-53c98e940de8 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:27.631399 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 252/498] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:27.631399 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c6c1e-ecad-4b13-8870-53c98e940de8 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:27.637427 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f45279-1234-420a-89a9-d9c55b645c98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:27.645861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-16f45279-1234-420a-89a9-d9c55b645c98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:27.645861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:27.645861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:27.645861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:27.645861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:27.645861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:27.645861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:27.645861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:27.645861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:27.645861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:27.645861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:27.645861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:27.645861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:27.645861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:27.645861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:27.645861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:27.646845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:27.646845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:27.646845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:27.646845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:27.646845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:27.646845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:27.646845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:27.646845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:27.646845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:27.646845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:27.646845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:27.646845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:27.646845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:27.646845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:27.646845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:27.646845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:27.647694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:27.647694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:27.647694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:27.647694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:27.647694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:27.647694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:27.647694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:27.647694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:27.647694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:27.647694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:27.647694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:27.647694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:27.647694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 247/499] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:27.648383 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ff89a-d884-423e-b85a-01226ea4c9d9 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:27.648619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ff89a-d884-423e-b85a-01226ea4c9d9 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:27.648838 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ff89a-d884-423e-b85a-01226ea4c9d9 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:27.650073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 248/500] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:27.650747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ff89a-d884-423e-b85a-01226ea4c9d9 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:27.743414 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c6c1e-ecad-4b13-8870-53c98e940de8 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:27.754394 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ff89a-d884-423e-b85a-01226ea4c9d9 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:27.773862 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:27.779002 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1dfbf7-0985-4467-91ca-bb7f6559be54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:27.779212 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:27.779852 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1dfbf7-0985-4467-91ca-bb7f6559be54 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:27.780114 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1dfbf7-0985-4467-91ca-bb7f6559be54 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:27.780452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1dfbf7-0985-4467-91ca-bb7f6559be54 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:27.788360 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1dfbf7-0985-4467-91ca-bb7f6559be54 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:27.789819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1dfbf7-0985-4467-91ca-bb7f6559be54 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:27.791131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b11d57-13eb-4cb1-b7b3-3d12ceb58fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:27.792061 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b11d57-13eb-4cb1-b7b3-3d12ceb58fc5 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:27.792353 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b11d57-13eb-4cb1-b7b3-3d12ceb58fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:27.792636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b11d57-13eb-4cb1-b7b3-3d12ceb58fc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:27.803484 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b11d57-13eb-4cb1-b7b3-3d12ceb58fc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:27.804902 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b11d57-13eb-4cb1-b7b3-3d12ceb58fc5 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:27.805647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-ad1dfbf7-0985-4467-91ca-bb7f6559be54 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 22 11:32:27.805647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:27.805647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:27.805647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:27.805647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:27.805647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:27.805647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:27.805647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:27.805647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:27.805647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:27.805647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:27.805647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:27.805647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:27.805647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:27.805647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:27.805647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:27.806666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:27.806666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:27.806666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:27.806666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:27.806666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:27.806666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:27.806666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:27.806666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:27.806666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:27.806666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:27.806666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:27.806666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:27.806666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:27.806666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:27.806666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:27.806666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:27.806666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:27.806666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:27.812711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:27.812711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:27.812711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:27.812711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:27.812711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:27.812711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:27.812711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:27.812711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:27.812711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 22 11:32:27.812711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:27.812711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 253/501] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:27.812711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117e451-7bb9-4095-a7ef-b348b6212d81 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:27.812711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117e451-7bb9-4095-a7ef-b348b6212d81 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:27.813536 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117e451-7bb9-4095-a7ef-b348b6212d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:27.827449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-48b11d57-13eb-4cb1-b7b3-3d12ceb58fc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:27.827449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:27.827449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:27.827449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:27.827449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:27.827449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:27.827449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:27.827449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:27.827449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:27.827449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:27.827449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:27.827449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:27.827449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:27.827449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:27.827449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:27.828371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:27.828371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:27.828371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:27.828371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:27.828371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:27.828371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:27.828371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:27.828371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:27.828371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:27.828371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:27.828371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:27.828371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:27.828371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:27.828371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:27.828371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:27.828371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:27.828371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:27.829242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:27.829242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:27.829242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:27.829242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:27.829242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 249/502] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:27.829498 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b11d57-13eb-4cb1-b7b3-3d12ceb58fc5 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:27.830590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c117e451-7bb9-4095-a7ef-b348b6212d81 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:27.831882 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c117e451-7bb9-4095-a7ef-b348b6212d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:27.902841 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 254/503] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:27.904160 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006e1e4-6d61-4374-b9b1-f11a663db2b9 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:27.904452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006e1e4-6d61-4374-b9b1-f11a663db2b9 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:27.904728 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006e1e4-6d61-4374-b9b1-f11a663db2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:27.922026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-0006e1e4-6d61-4374-b9b1-f11a663db2b9 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:27.923642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-0006e1e4-6d61-4374-b9b1-f11a663db2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:27.950565 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b11d57-13eb-4cb1-b7b3-3d12ceb58fc5 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:27.984490 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ce665-ea5d-4d62-925b-1e1d5fa6a23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:27.985153 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ce665-ea5d-4d62-925b-1e1d5fa6a23d None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:27.985441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ce665-ea5d-4d62-925b-1e1d5fa6a23d None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:27.985724 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ce665-ea5d-4d62-925b-1e1d5fa6a23d None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:27.986485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ce665-ea5d-4d62-925b-1e1d5fa6a23d None admin] RBAC: Adding query filter params (name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:27.986790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ce665-ea5d-4d62-925b-1e1d5fa6a23d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:27.988714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ce665-ea5d-4d62-925b-1e1d5fa6a23d None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:27.995863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 255/504] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:27.996481 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006e1e4-6d61-4374-b9b1-f11a663db2b9 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:28.001078 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 250/505] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:32:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:28.001078 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ce665-ea5d-4d62-925b-1e1d5fa6a23d None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:28.039067 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:28.057025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:28.103000 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ce665-ea5d-4d62-925b-1e1d5fa6a23d None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:28.103251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006e1e4-6d61-4374-b9b1-f11a663db2b9 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:28.136377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b08960-678c-488a-8ff7-1bce4f578408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:28.136594 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3d057-d031-45d8-a690-6c166fd00960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:28.137037 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b08960-678c-488a-8ff7-1bce4f578408 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:28.137217 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3d057-d031-45d8-a690-6c166fd00960 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:28.137305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b08960-678c-488a-8ff7-1bce4f578408 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:28.137499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3d057-d031-45d8-a690-6c166fd00960 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:28.137588 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b08960-678c-488a-8ff7-1bce4f578408 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:28.137772 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3d057-d031-45d8-a690-6c166fd00960 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:28.138291 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b08960-678c-488a-8ff7-1bce4f578408 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:28.138598 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3d057-d031-45d8-a690-6c166fd00960 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:28.138692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b08960-678c-488a-8ff7-1bce4f578408 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:28.138901 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3d057-d031-45d8-a690-6c166fd00960 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:28.140268 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b08960-678c-488a-8ff7-1bce4f578408 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:28.140598 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3d057-d031-45d8-a690-6c166fd00960 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:28.152675 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 251/506] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:28.153484 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b08960-678c-488a-8ff7-1bce4f578408 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:28.177987 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 256/507] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jun 22 11:32:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:28.178846 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3d057-d031-45d8-a690-6c166fd00960 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:28.249140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:28.255296 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b08960-678c-488a-8ff7-1bce4f578408 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:28.280233 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3d057-d031-45d8-a690-6c166fd00960 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:28.291120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da1fcd-ea3c-4dbf-84ef-811053eb6489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:28.291747 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:28.291947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da1fcd-ea3c-4dbf-84ef-811053eb6489 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:28.292232 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da1fcd-ea3c-4dbf-84ef-811053eb6489 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:28.292510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da1fcd-ea3c-4dbf-84ef-811053eb6489 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:28.303175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da1fcd-ea3c-4dbf-84ef-811053eb6489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:28.310780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-53da1fcd-ea3c-4dbf-84ef-811053eb6489 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:28.310780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:28.310780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:28.310780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:28.310780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:28.310780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:28.310780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:28.310780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:28.310780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:28.310780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:28.310780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:28.310780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:28.310780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:28.310780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:28.310780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:28.310780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:28.311849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:28.311849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:28.311849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:28.311849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:28.311849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:28.311849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:28.311849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:28.311849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:28.311849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:28.311849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:28.311849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:28.311849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:28.311849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:28.311849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:28.311849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:28.311849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:28.313152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:28.313152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:28.313152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:28.313152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:28.313152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:28.313152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:28.313152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:28.313152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:28.313152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:28.313152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:28.313152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:28.313152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:28.313152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 252/508] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:28.313152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da1fcd-ea3c-4dbf-84ef-811053eb6489 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:28.319597 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437db03b-4c8b-4b53-8ffe-d9b5a805f6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:28.320395 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437db03b-4c8b-4b53-8ffe-d9b5a805f6d6 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:28.320675 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437db03b-4c8b-4b53-8ffe-d9b5a805f6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:28.320948 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437db03b-4c8b-4b53-8ffe-d9b5a805f6d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:28.331191 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437db03b-4c8b-4b53-8ffe-d9b5a805f6d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:28.333206 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437db03b-4c8b-4b53-8ffe-d9b5a805f6d6 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:28.350394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-437db03b-4c8b-4b53-8ffe-d9b5a805f6d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:28.350394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:28.350394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:28.350394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:28.350394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:28.350394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:28.350394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:28.350394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:28.350394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:28.350394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:28.350394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:28.350394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:28.350394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:28.350394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:28.350394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:28.351439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:28.351439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:28.351439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:28.351439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:28.351439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:28.351439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:28.351439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:28.351439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:28.351439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:28.351439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:28.351439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:28.351439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:28.351439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:28.351439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:28.351439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:28.351439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:28.351439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:28.352307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:28.352307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:28.352307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:28.352307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:28.352307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 257/509] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:28.352771 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437db03b-4c8b-4b53-8ffe-d9b5a805f6d6 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:28.421395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da1fcd-ea3c-4dbf-84ef-811053eb6489 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:28.452592 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437db03b-4c8b-4b53-8ffe-d9b5a805f6d6 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:28.454958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451f2a2-4baa-4175-aca7-3183e7794fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:28.455801 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451f2a2-4baa-4175-aca7-3183e7794fd6 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:28.456086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451f2a2-4baa-4175-aca7-3183e7794fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:28.456364 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451f2a2-4baa-4175-aca7-3183e7794fd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:28.466811 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451f2a2-4baa-4175-aca7-3183e7794fd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:28.468344 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451f2a2-4baa-4175-aca7-3183e7794fd6 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:28.485849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-5451f2a2-4baa-4175-aca7-3183e7794fd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:28.485849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:28.485849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:28.485849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:28.485849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:28.485849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:28.485849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:28.485849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:28.485849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:28.485849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:28.485849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:28.485849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:28.485849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:28.485849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:28.485849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:28.486914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:28.486914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:28.486914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:28.486914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:28.486914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:28.486914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:28.486914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:28.486914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:28.486914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:28.486914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:28.486914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:28.486914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:28.486914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:28.486914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:28.486914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:28.486914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:28.486914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:28.487840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:28.487840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:28.487840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:28.487840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:28.488055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70061cff-4a4c-426e-b9d9-e3aaf852a193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:28.488674 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 253/510] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:28.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70061cff-4a4c-426e-b9d9-e3aaf852a193 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:28.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70061cff-4a4c-426e-b9d9-e3aaf852a193 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:28.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70061cff-4a4c-426e-b9d9-e3aaf852a193 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:28.488941 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451f2a2-4baa-4175-aca7-3183e7794fd6 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:28.495597 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70061cff-4a4c-426e-b9d9-e3aaf852a193 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:28.497053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70061cff-4a4c-426e-b9d9-e3aaf852a193 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:28.500185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:28.512072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-70061cff-4a4c-426e-b9d9-e3aaf852a193 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:28.512072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:28.512072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:28.512072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:28.512072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:28.512072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:28.512072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:28.512072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:28.512072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:28.512072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:28.512072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:28.512072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:28.512072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:28.512072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:28.512072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:28.512961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:28.512961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:28.512961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:28.512961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:28.512961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:28.512961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:28.512961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:28.512961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:28.512961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:28.512961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:28.512961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:28.512961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:28.512961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:28.512961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:28.512961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:28.512961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:28.512961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:28.513817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:28.513817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:28.513817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:28.513817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:28.513817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:28.513817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:28.513817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:28.513817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:28.513817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 258/511] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:28.514280 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70061cff-4a4c-426e-b9d9-e3aaf852a193 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:28.538840 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:28.595216 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451f2a2-4baa-4175-aca7-3183e7794fd6 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:28.618766 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70061cff-4a4c-426e-b9d9-e3aaf852a193 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:28.634202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad18fc3-28c9-4fe6-a71f-57c7eac42162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:28.634863 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad18fc3-28c9-4fe6-a71f-57c7eac42162 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:28.635145 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad18fc3-28c9-4fe6-a71f-57c7eac42162 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:28.635447 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad18fc3-28c9-4fe6-a71f-57c7eac42162 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:28.662034 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527f9aa-b453-4209-a8ad-907957580cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:28.662694 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527f9aa-b453-4209-a8ad-907957580cf6 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:28.662970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527f9aa-b453-4209-a8ad-907957580cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:28.663483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527f9aa-b453-4209-a8ad-907957580cf6 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:28.664790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527f9aa-b453-4209-a8ad-907957580cf6 None admin] RBAC: Adding query filter params (name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:28.665779 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527f9aa-b453-4209-a8ad-907957580cf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:28.666875 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527f9aa-b453-4209-a8ad-907957580cf6 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:28.676351 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 259/512] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:32:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:28.677141 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527f9aa-b453-4209-a8ad-907957580cf6 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:28.705141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad18fc3-28c9-4fe6-a71f-57c7eac42162 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:28.706612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad18fc3-28c9-4fe6-a71f-57c7eac42162 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:28.717187 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:28.761162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:28.780275 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527f9aa-b453-4209-a8ad-907957580cf6 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:28.811002 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 254/513] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:28.812189 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad18fc3-28c9-4fe6-a71f-57c7eac42162 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:28.821881 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc16a63-0846-4649-bff9-169911880058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:28.822549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc16a63-0846-4649-bff9-169911880058 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:28.822830 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc16a63-0846-4649-bff9-169911880058 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:28.823150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc16a63-0846-4649-bff9-169911880058 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:28.823871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc16a63-0846-4649-bff9-169911880058 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:28.824175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc16a63-0846-4649-bff9-169911880058 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:28.825548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc16a63-0846-4649-bff9-169911880058 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:28.836878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 260/514] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:28.837693 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc16a63-0846-4649-bff9-169911880058 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:28.916210 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad18fc3-28c9-4fe6-a71f-57c7eac42162 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:28.926084 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:28.938927 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc16a63-0846-4649-bff9-169911880058 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:28.958570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c88ae7-ca0a-489a-ad80-6b98a8661b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:28.959240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c88ae7-ca0a-489a-ad80-6b98a8661b9d None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:28.959835 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c88ae7-ca0a-489a-ad80-6b98a8661b9d None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:28.960131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c88ae7-ca0a-489a-ad80-6b98a8661b9d None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:28.960861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c88ae7-ca0a-489a-ad80-6b98a8661b9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:28.961156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c88ae7-ca0a-489a-ad80-6b98a8661b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:28.962551 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c88ae7-ca0a-489a-ad80-6b98a8661b9d None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:28.973172 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445399e-55c5-45b2-9522-1917a7f444b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:28.973830 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445399e-55c5-45b2-9522-1917a7f444b2 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:28.974368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445399e-55c5-45b2-9522-1917a7f444b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:28.974474 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 255/515] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:28.974570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445399e-55c5-45b2-9522-1917a7f444b2 None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:28.974996 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c88ae7-ca0a-489a-ad80-6b98a8661b9d None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:28.975156 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445399e-55c5-45b2-9522-1917a7f444b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:28.975496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445399e-55c5-45b2-9522-1917a7f444b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:28.976919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2445399e-55c5-45b2-9522-1917a7f444b2 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:28.995182 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 261/516] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:28.996026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445399e-55c5-45b2-9522-1917a7f444b2 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:29.002536 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:29.078181 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c88ae7-ca0a-489a-ad80-6b98a8661b9d None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:29.098182 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445399e-55c5-45b2-9522-1917a7f444b2 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:29.111273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd01c5d-0554-4c70-bb53-862b1e0b75dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:29.112089 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd01c5d-0554-4c70-bb53-862b1e0b75dc None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:29.112378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd01c5d-0554-4c70-bb53-862b1e0b75dc None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:29.112661 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd01c5d-0554-4c70-bb53-862b1e0b75dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:29.124793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd01c5d-0554-4c70-bb53-862b1e0b75dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:29.126265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd01c5d-0554-4c70-bb53-862b1e0b75dc None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:29.132943 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9ba9e-a8ed-47bf-bb71-9a06308108f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:29.133720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9ba9e-a8ed-47bf-bb71-9a06308108f4 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:29.134000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9ba9e-a8ed-47bf-bb71-9a06308108f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:29.134281 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9ba9e-a8ed-47bf-bb71-9a06308108f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:29.143466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-0cd01c5d-0554-4c70-bb53-862b1e0b75dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:29.143466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:29.143466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:29.143466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:29.143466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:29.143466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:29.143466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:29.143466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:29.143466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:29.143466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:29.143466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:29.143466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:29.143466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:29.143466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:29.143466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:29.144308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:29.144308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:29.144308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:29.144308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:29.144308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:29.144308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:29.144308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:29.144308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:29.144308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:29.144308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:29.144308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:29.144308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:29.144308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:29.144308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:29.144308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:29.144308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:29.144308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:29.145169 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:29.145169 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:29.145169 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:29.145169 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:29.145374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9ba9e-a8ed-47bf-bb71-9a06308108f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:29.145441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 256/517] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:29.145441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd01c5d-0554-4c70-bb53-862b1e0b75dc None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:29.146106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9ba9e-a8ed-47bf-bb71-9a06308108f4 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:29.167972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-b3b9ba9e-a8ed-47bf-bb71-9a06308108f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:29.167972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:29.167972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:29.167972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:29.167972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:29.167972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:29.167972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:29.167972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:29.167972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:29.167972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:29.167972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:29.167972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:29.167972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:29.167972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:29.167972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:29.168908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:29.168908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:29.168908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:29.168908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:29.168908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:29.168908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:29.168908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:29.168908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:29.168908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:29.168908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:29.168908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:29.168908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:29.168908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:29.168908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:29.168908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:29.168908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:29.168908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:29.170064 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:29.170064 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:29.170064 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:29.170064 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:29.170064 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 262/518] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:29.170506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9ba9e-a8ed-47bf-bb71-9a06308108f4 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:29.174379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:29.226585 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:29.258169 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd01c5d-0554-4c70-bb53-862b1e0b75dc None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:29.270692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9ba9e-a8ed-47bf-bb71-9a06308108f4 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:29.292334 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1aa4c-bb0c-4c9b-bc0e-daa3f2642dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:29.293134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1aa4c-bb0c-4c9b-bc0e-daa3f2642dad None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:29.293424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1aa4c-bb0c-4c9b-bc0e-daa3f2642dad None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:29.293701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1aa4c-bb0c-4c9b-bc0e-daa3f2642dad None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:29.301156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1aa4c-bb0c-4c9b-bc0e-daa3f2642dad None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:29.302610 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1aa4c-bb0c-4c9b-bc0e-daa3f2642dad None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:29.303754 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70664f1-036a-4615-9c48-117f7f63a76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:29.304414 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70664f1-036a-4615-9c48-117f7f63a76c None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:29.304690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70664f1-036a-4615-9c48-117f7f63a76c None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:29.304963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70664f1-036a-4615-9c48-117f7f63a76c None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:29.305658 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70664f1-036a-4615-9c48-117f7f63a76c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:29.307078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70664f1-036a-4615-9c48-117f7f63a76c None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:29.317421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-92f1aa4c-bb0c-4c9b-bc0e-daa3f2642dad None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 22 11:32:29.317421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:29.317421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:29.317421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:29.317421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:29.317421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:29.317421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:29.317421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:29.317421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:29.317421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:29.317421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:29.317421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:29.317421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:29.317421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:29.317421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:29.317421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:29.318636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:29.318636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:29.318636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:29.318636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:29.318636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:29.318636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:29.318636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:29.318636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:29.318636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:29.318636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:29.318636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:29.318636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:29.318636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:29.318636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:29.318636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:29.318636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:29.318636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:29.318636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:29.319557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:29.319557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:29.319557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:29.319557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:29.319557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:29.319557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:29.319557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:29.319557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:29.319557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 22 11:32:29.319557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:29.319557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 257/519] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:29.320143 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1aa4c-bb0c-4c9b-bc0e-daa3f2642dad None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:29.343472 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 263/520] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jun 22 11:32:29 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:32:29.343472 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70664f1-036a-4615-9c48-117f7f63a76c None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:29.422897 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:29.445630 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1aa4c-bb0c-4c9b-bc0e-daa3f2642dad None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:29.446781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70664f1-036a-4615-9c48-117f7f63a76c None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:29.462621 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:29.484462 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b81ea7-d095-4321-8745-ea15fb596640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:29.485136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b81ea7-d095-4321-8745-ea15fb596640 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:29.485427 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b81ea7-d095-4321-8745-ea15fb596640 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:29.485700 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b81ea7-d095-4321-8745-ea15fb596640 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:29.486555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b81ea7-d095-4321-8745-ea15fb596640 None admin] RBAC: Adding query filter params (role.id=64c2d69f15ca458c8267c1acfd42d9fc, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=479a376dcac14c509c1c55f0046d88cf) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:29.486994 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b81ea7-d095-4321-8745-ea15fb596640 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:29.488911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b81ea7-d095-4321-8745-ea15fb596640 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:29.492043 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540cf8d-10f2-4a1e-8e1e-0dadb845b4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:29.492716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540cf8d-10f2-4a1e-8e1e-0dadb845b4b6 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:29.493650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540cf8d-10f2-4a1e-8e1e-0dadb845b4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:29.493650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540cf8d-10f2-4a1e-8e1e-0dadb845b4b6 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:29.494015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540cf8d-10f2-4a1e-8e1e-0dadb845b4b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:29.494288 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540cf8d-10f2-4a1e-8e1e-0dadb845b4b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:29.495689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540cf8d-10f2-4a1e-8e1e-0dadb845b4b6 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:29.496448 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 258/521] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:32:29 2022] GET /identity/v3/role_assignments?user.id=479a376dcac14c509c1c55f0046d88cf&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=64c2d69f15ca458c8267c1acfd42d9fc => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:29.497001 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b81ea7-d095-4321-8745-ea15fb596640 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:29.512714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 264/522] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:29.513479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540cf8d-10f2-4a1e-8e1e-0dadb845b4b6 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:29.635088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b81ea7-d095-4321-8745-ea15fb596640 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:29.656318 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540cf8d-10f2-4a1e-8e1e-0dadb845b4b6 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:29.673619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b96028-288d-4ec6-a330-16037343460d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:29.674648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b96028-288d-4ec6-a330-16037343460d None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:29.675123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b96028-288d-4ec6-a330-16037343460d None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:29.675762 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b96028-288d-4ec6-a330-16037343460d None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:29.676690 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b96028-288d-4ec6-a330-16037343460d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:29.677206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b96028-288d-4ec6-a330-16037343460d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:29.678854 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b96028-288d-4ec6-a330-16037343460d None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:29.692169 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 259/523] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:29.693485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b96028-288d-4ec6-a330-16037343460d None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:29.695659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd707907-68f8-439e-a826-b17baaf5782a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:29.696446 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd707907-68f8-439e-a826-b17baaf5782a None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:29.696941 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd707907-68f8-439e-a826-b17baaf5782a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:29.697316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd707907-68f8-439e-a826-b17baaf5782a None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:29.699136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:29.699262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd707907-68f8-439e-a826-b17baaf5782a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:29.699465 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd707907-68f8-439e-a826-b17baaf5782a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:29.701268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd707907-68f8-439e-a826-b17baaf5782a None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:29.723397 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 265/524] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Jun 22 11:32:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:29.724319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd707907-68f8-439e-a826-b17baaf5782a None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:29.758023 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:29.801812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b96028-288d-4ec6-a330-16037343460d None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:29.839690 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf0c4f-2a1f-4d1a-bf1a-4878bed848a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:29.840491 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf0c4f-2a1f-4d1a-bf1a-4878bed848a1 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:29.840774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf0c4f-2a1f-4d1a-bf1a-4878bed848a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:29.840901 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd707907-68f8-439e-a826-b17baaf5782a None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:29.841304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf0c4f-2a1f-4d1a-bf1a-4878bed848a1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:29.849510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf0c4f-2a1f-4d1a-bf1a-4878bed848a1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:29.851006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf0c4f-2a1f-4d1a-bf1a-4878bed848a1 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:29.867426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-63bf0c4f-2a1f-4d1a-bf1a-4878bed848a1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 22 11:32:29.867426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:29.867426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:29.867426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:29.867426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:29.867426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:29.867426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:29.867426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:29.867426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:29.867426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:29.867426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:29.867426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:29.867426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:29.867426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:29.867426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:29.867426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:29.868417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:29.868417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:29.868417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:29.868417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:29.868417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:29.868417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:29.868417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:29.868417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:29.868417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:29.868417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:29.868417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:29.868417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:29.868417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:29.868417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:29.868417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:29.868417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:29.868417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:29.868417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:29.869377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:29.869377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:29.869377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:29.869377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:29.869377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:29.869377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:29.869377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:29.869377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:29.869377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 22 11:32:29.869377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:29.869377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 260/525] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:29.869942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf0c4f-2a1f-4d1a-bf1a-4878bed848a1 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:29.884056 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3daae6-b3ab-417d-8ece-2f85d7cc3928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:29.884996 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3daae6-b3ab-417d-8ece-2f85d7cc3928 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:29.885336 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3daae6-b3ab-417d-8ece-2f85d7cc3928 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:29.885558 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3daae6-b3ab-417d-8ece-2f85d7cc3928 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:29.893720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3daae6-b3ab-417d-8ece-2f85d7cc3928 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:29.895395 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3daae6-b3ab-417d-8ece-2f85d7cc3928 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:29.911655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-1f3daae6-b3ab-417d-8ece-2f85d7cc3928 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:29.911655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:29.911655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:29.911655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:29.911655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:29.911655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:29.911655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:29.911655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:29.911655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:29.911655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:29.911655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:29.911655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:29.911655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:29.911655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:29.911655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:29.912567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:29.912567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:29.912567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:29.912567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:29.912567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:29.912567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:29.912567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:29.912567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:29.912567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:29.912567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:29.912567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:29.912567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:29.912567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:29.912567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:29.912567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:29.912567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:29.912567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:29.913400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:29.913400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:29.913400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:29.913400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:29.913400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:29.913400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:29.913400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:29.913400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:29.919591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 266/526] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:29.920711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3daae6-b3ab-417d-8ece-2f85d7cc3928 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:29.944429 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:29.987592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf0c4f-2a1f-4d1a-bf1a-4878bed848a1 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:30.025798 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec4ee3-5e5f-4f4c-a2ca-c6d96430d62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:30.026495 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec4ee3-5e5f-4f4c-a2ca-c6d96430d62e None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:30.026790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec4ee3-5e5f-4f4c-a2ca-c6d96430d62e None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:30.027070 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec4ee3-5e5f-4f4c-a2ca-c6d96430d62e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:30.044804 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:30.059438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3daae6-b3ab-417d-8ece-2f85d7cc3928 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:30.102141 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdb2ec-5257-47ed-8087-d1392027fb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:30.102840 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdb2ec-5257-47ed-8087-d1392027fb20 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:30.103119 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdb2ec-5257-47ed-8087-d1392027fb20 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:30.103422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cdb2ec-5257-47ed-8087-d1392027fb20 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:30.104377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cdb2ec-5257-47ed-8087-d1392027fb20 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:30.104682 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cdb2ec-5257-47ed-8087-d1392027fb20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:30.106257 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cdb2ec-5257-47ed-8087-d1392027fb20 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:30.110155 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec4ee3-5e5f-4f4c-a2ca-c6d96430d62e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:30.111914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec4ee3-5e5f-4f4c-a2ca-c6d96430d62e None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:30.124455 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 267/527] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jun 22 11:32:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:30.125675 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdb2ec-5257-47ed-8087-d1392027fb20 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:30.181262 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:30.227552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 261/528] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:30.232814 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec4ee3-5e5f-4f4c-a2ca-c6d96430d62e None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:30.258447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cdb2ec-5257-47ed-8087-d1392027fb20 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:30.296056 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e5899-cebe-4ba7-8770-df4ac3e6017b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:30.296724 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e5899-cebe-4ba7-8770-df4ac3e6017b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:30.297007 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e5899-cebe-4ba7-8770-df4ac3e6017b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:30.297281 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e5899-cebe-4ba7-8770-df4ac3e6017b None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:30.298265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e5899-cebe-4ba7-8770-df4ac3e6017b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:30.298585 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e5899-cebe-4ba7-8770-df4ac3e6017b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:30.300047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e5899-cebe-4ba7-8770-df4ac3e6017b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:30.313303 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 268/529] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:30.314239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e5899-cebe-4ba7-8770-df4ac3e6017b None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:30.337939 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec4ee3-5e5f-4f4c-a2ca-c6d96430d62e None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:30.343693 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:30.372967 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88475ea5-e28f-47ce-b7bc-4e7b2ce1e34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:30.373644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88475ea5-e28f-47ce-b7bc-4e7b2ce1e34b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:30.373925 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88475ea5-e28f-47ce-b7bc-4e7b2ce1e34b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:30.374204 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88475ea5-e28f-47ce-b7bc-4e7b2ce1e34b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:30.406272 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:30.424138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e5899-cebe-4ba7-8770-df4ac3e6017b None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:30.445159 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88475ea5-e28f-47ce-b7bc-4e7b2ce1e34b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:30.446552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88475ea5-e28f-47ce-b7bc-4e7b2ce1e34b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:30.462564 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28674359-1f8b-4daf-b93d-8c124dd281ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:30.463371 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28674359-1f8b-4daf-b93d-8c124dd281ca None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:30.463714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28674359-1f8b-4daf-b93d-8c124dd281ca None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:30.464091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28674359-1f8b-4daf-b93d-8c124dd281ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:30.475529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28674359-1f8b-4daf-b93d-8c124dd281ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:30.477240 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28674359-1f8b-4daf-b93d-8c124dd281ca None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:30.497729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-28674359-1f8b-4daf-b93d-8c124dd281ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:30.497729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:30.497729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:30.497729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:30.497729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:30.497729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:30.497729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:30.497729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:30.497729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:30.497729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:30.497729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:30.497729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:30.497729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:30.497729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:30.497729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:30.498629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:30.498629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:30.498629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:30.498629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:30.498629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:30.498629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:30.498629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:30.498629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:30.498629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:30.498629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:30.498629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:30.498629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:30.498629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:30.498629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:30.498629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:30.498629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:30.498629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:30.499526 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:30.499526 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:30.499526 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:30.499526 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:30.499526 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 269/530] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:30.500259 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28674359-1f8b-4daf-b93d-8c124dd281ca None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:30.553376 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 262/531] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:30.554132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88475ea5-e28f-47ce-b7bc-4e7b2ce1e34b None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:30.573393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:30.628044 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:30.628271 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28674359-1f8b-4daf-b93d-8c124dd281ca None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:30.675737 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025fe9dd-2df6-4fc9-aedb-9b58119eb988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:30.676399 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fe9dd-2df6-4fc9-aedb-9b58119eb988 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:30.676997 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fe9dd-2df6-4fc9-aedb-9b58119eb988 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:30.677273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fe9dd-2df6-4fc9-aedb-9b58119eb988 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:30.678383 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025fe9dd-2df6-4fc9-aedb-9b58119eb988 None admin] RBAC: Adding query filter params (role.id=64c2d69f15ca458c8267c1acfd42d9fc, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=7ed133721ed044708ee362c93f564d07) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:30.678693 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025fe9dd-2df6-4fc9-aedb-9b58119eb988 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:30.680342 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025fe9dd-2df6-4fc9-aedb-9b58119eb988 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:30.683555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88475ea5-e28f-47ce-b7bc-4e7b2ce1e34b None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:30.687225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 270/532] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:32:30 2022] GET /identity/v3/role_assignments?user.id=7ed133721ed044708ee362c93f564d07&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=64c2d69f15ca458c8267c1acfd42d9fc => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:30.688932 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025fe9dd-2df6-4fc9-aedb-9b58119eb988 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:30.748261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ba5d0-67c7-443d-b733-eddaa94c09b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:30.750065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ba5d0-67c7-443d-b733-eddaa94c09b3 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:30.751216 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ba5d0-67c7-443d-b733-eddaa94c09b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:30.752546 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ba5d0-67c7-443d-b733-eddaa94c09b3 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:30.753597 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ba5d0-67c7-443d-b733-eddaa94c09b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:30.756635 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ba5d0-67c7-443d-b733-eddaa94c09b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:30.758379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ba5d0-67c7-443d-b733-eddaa94c09b3 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:30.788805 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 263/533] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:30.791599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ba5d0-67c7-443d-b733-eddaa94c09b3 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:30.897235 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025fe9dd-2df6-4fc9-aedb-9b58119eb988 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:30.925148 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:30.947335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8420e8-59f5-473f-8644-4cd5ae629c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:30.948585 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8420e8-59f5-473f-8644-4cd5ae629c5b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:30.951203 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8420e8-59f5-473f-8644-4cd5ae629c5b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:30.951767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8420e8-59f5-473f-8644-4cd5ae629c5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:30.970106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8420e8-59f5-473f-8644-4cd5ae629c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:30.973086 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8420e8-59f5-473f-8644-4cd5ae629c5b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:30.986898 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ba5d0-67c7-443d-b733-eddaa94c09b3 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:30.996576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-6a8420e8-59f5-473f-8644-4cd5ae629c5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:30.996576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:30.996576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:30.996576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:30.996576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:30.996576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:30.996576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:30.996576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:30.996576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:30.996576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:30.996576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:30.996576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:30.996576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:30.996576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:30.996576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:30.997457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:30.997457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:30.997457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:30.997457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:30.997457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:30.997457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:30.997457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:30.997457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:30.997457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:30.997457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:30.997457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:30.997457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:30.997457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:30.997457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:30.997457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:30.997457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:30.997457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:30.998359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:30.998359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:30.998359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:30.998359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:31.000738 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 271/534] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:31.006222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8420e8-59f5-473f-8644-4cd5ae629c5b None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:31.031913 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:31.053273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b1b14-05f2-4093-b58f-42980cc51bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:31.054444 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b1b14-05f2-4093-b58f-42980cc51bbf None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:31.054935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b1b14-05f2-4093-b58f-42980cc51bbf None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:31.055458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b1b14-05f2-4093-b58f-42980cc51bbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:31.070303 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b1b14-05f2-4093-b58f-42980cc51bbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:31.072172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b1b14-05f2-4093-b58f-42980cc51bbf None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:31.101922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-4a3b1b14-05f2-4093-b58f-42980cc51bbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:31.101922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:31.101922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:31.101922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:31.101922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:31.101922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:31.101922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:31.101922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:31.101922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:31.101922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:31.101922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:31.101922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:31.101922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:31.101922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:31.101922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:31.102837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:31.102837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:31.102837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:31.102837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:31.102837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:31.102837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:31.102837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:31.102837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:31.102837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:31.102837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:31.102837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:31.102837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:31.102837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:31.102837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:31.102837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:31.102837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:31.102837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:31.103733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:31.103733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:31.103733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:31.103733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:31.103733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:31.103733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:31.103733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:31.103733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:31.106273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 264/535] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:31.112188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b1b14-05f2-4093-b58f-42980cc51bbf None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:31.219545 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8420e8-59f5-473f-8644-4cd5ae629c5b None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:31.259128 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1bc084-9208-43db-afdc-9efb3279a507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:31.260155 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bc084-9208-43db-afdc-9efb3279a507 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:31.260663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bc084-9208-43db-afdc-9efb3279a507 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:31.261137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bc084-9208-43db-afdc-9efb3279a507 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:31.262120 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1bc084-9208-43db-afdc-9efb3279a507 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:31.262651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1bc084-9208-43db-afdc-9efb3279a507 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:31.264270 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1bc084-9208-43db-afdc-9efb3279a507 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:31.277803 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 272/536] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:31.283371 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1bc084-9208-43db-afdc-9efb3279a507 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:31.320549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:31.353020 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b1b14-05f2-4093-b58f-42980cc51bbf None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:31.428292 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1bc084-9208-43db-afdc-9efb3279a507 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:31.428481 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d8762-096a-47e6-b533-087e914cb3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:31.429142 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d8762-096a-47e6-b533-087e914cb3b9 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:31.429430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d8762-096a-47e6-b533-087e914cb3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:31.429706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d8762-096a-47e6-b533-087e914cb3b9 None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:31.430498 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d8762-096a-47e6-b533-087e914cb3b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:31.430807 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d8762-096a-47e6-b533-087e914cb3b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:31.432265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d8762-096a-47e6-b533-087e914cb3b9 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:31.463692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bbdec-e79a-4a61-a2ac-7f3ec0ed685d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:31.464491 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bbdec-e79a-4a61-a2ac-7f3ec0ed685d None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:31.464781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bbdec-e79a-4a61-a2ac-7f3ec0ed685d None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:31.465064 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040bbdec-e79a-4a61-a2ac-7f3ec0ed685d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:31.470265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 265/537] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:31.472919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bbdec-e79a-4a61-a2ac-7f3ec0ed685d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:31.474406 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040bbdec-e79a-4a61-a2ac-7f3ec0ed685d None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:31.476774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d8762-096a-47e6-b533-087e914cb3b9 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:31.488781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-040bbdec-e79a-4a61-a2ac-7f3ec0ed685d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:31.488781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:31.488781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:31.488781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:31.488781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:31.488781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:31.488781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:31.488781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:31.488781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:31.488781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:31.488781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:31.488781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:31.488781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:31.488781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:31.488781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:31.489720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:31.489720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:31.489720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:31.489720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:31.489720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:31.489720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:31.489720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:31.489720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:31.489720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:31.489720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:31.489720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:31.489720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:31.489720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:31.489720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:31.489720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:31.489720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:31.489720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:31.490567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:31.490567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:31.490567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:31.490567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:31.490567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:31.490567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:31.490567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:31.490567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:31.490567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 273/538] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:31.495828 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bbdec-e79a-4a61-a2ac-7f3ec0ed685d None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:31.566991 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:31.598956 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:31.604045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040bbdec-e79a-4a61-a2ac-7f3ec0ed685d None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:31.644282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e1a1b-32b2-46a8-a750-14cb6d167d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:31.644957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e1a1b-32b2-46a8-a750-14cb6d167d12 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:31.645242 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e1a1b-32b2-46a8-a750-14cb6d167d12 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:31.645522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e1a1b-32b2-46a8-a750-14cb6d167d12 None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:31.646319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e1a1b-32b2-46a8-a750-14cb6d167d12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:31.646630 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e1a1b-32b2-46a8-a750-14cb6d167d12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:31.648279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e1a1b-32b2-46a8-a750-14cb6d167d12 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:31.662565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 274/539] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:31.668865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb90269-5ccf-43c6-b730-c96b78af1e25 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:31.669162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb90269-5ccf-43c6-b730-c96b78af1e25 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:31.669444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb90269-5ccf-43c6-b730-c96b78af1e25 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:31.671515 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 275/540] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:31.672505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb90269-5ccf-43c6-b730-c96b78af1e25 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:31.685053 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d8762-096a-47e6-b533-087e914cb3b9 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:31.724887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb268e-ae23-4684-a1e2-1c857d7ac33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:31.725935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb268e-ae23-4684-a1e2-1c857d7ac33c None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:31.726482 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb268e-ae23-4684-a1e2-1c857d7ac33c None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:31.727065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb268e-ae23-4684-a1e2-1c857d7ac33c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:31.728270 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb268e-ae23-4684-a1e2-1c857d7ac33c None admin] RBAC: Adding query filter params (role.id=64c2d69f15ca458c8267c1acfd42d9fc, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=c00d0bb777a044a3a4d4540d187d938a) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:31.728810 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb268e-ae23-4684-a1e2-1c857d7ac33c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:31.730646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb268e-ae23-4684-a1e2-1c857d7ac33c None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:31.743377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 266/541] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:32:31 2022] GET /identity/v3/role_assignments?user.id=c00d0bb777a044a3a4d4540d187d938a&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=64c2d69f15ca458c8267c1acfd42d9fc => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:31.743377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cc9f3-ad13-4465-b60a-606c3c8a1204 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:31.743377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cc9f3-ad13-4465-b60a-606c3c8a1204 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:31.743377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cc9f3-ad13-4465-b60a-606c3c8a1204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:31.758451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-916cc9f3-ad13-4465-b60a-606c3c8a1204 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:31.759736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-916cc9f3-ad13-4465-b60a-606c3c8a1204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:31.804623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb90269-5ccf-43c6-b730-c96b78af1e25 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:31.839835 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 267/542] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:31.841516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973e5ba-1666-49b0-83ba-359f6a54f469 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:31.841921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973e5ba-1666-49b0-83ba-359f6a54f469 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:31.842294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1973e5ba-1666-49b0-83ba-359f6a54f469 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:31.848017 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 268/543] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:31.849977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283791a-bf4f-43a6-91fd-f79a01bfed7f None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:31.850263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283791a-bf4f-43a6-91fd-f79a01bfed7f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:31.850548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283791a-bf4f-43a6-91fd-f79a01bfed7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:31.854738 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0b61f-a573-4c2b-b30d-f5eae7b00fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:31.856087 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0b61f-a573-4c2b-b30d-f5eae7b00fb9 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:31.857655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0b61f-a573-4c2b-b30d-f5eae7b00fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:31.858187 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0b61f-a573-4c2b-b30d-f5eae7b00fb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:31.859374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0b61f-a573-4c2b-b30d-f5eae7b00fb9 None admin] RBAC: Adding query filter params (role.id=64c2d69f15ca458c8267c1acfd42d9fc, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=ceb170ac6353495b9d1dc6723d59567a) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:31.859986 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0b61f-a573-4c2b-b30d-f5eae7b00fb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:31.862579 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0b61f-a573-4c2b-b30d-f5eae7b00fb9 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:31.868201 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-7283791a-bf4f-43a6-91fd-f79a01bfed7f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:31.869407 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-7283791a-bf4f-43a6-91fd-f79a01bfed7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:31.870763 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 276/544] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:32:31 2022] GET /identity/v3/role_assignments?user.id=ceb170ac6353495b9d1dc6723d59567a&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=64c2d69f15ca458c8267c1acfd42d9fc => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:31.876761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0b61f-a573-4c2b-b30d-f5eae7b00fb9 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:31.909450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:31.963644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:32.037048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 269/545] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:32.062092 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0b61f-a573-4c2b-b30d-f5eae7b00fb9 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:32.074108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283791a-bf4f-43a6-91fd-f79a01bfed7f None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:32.115599 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a0ccb-5a6b-493a-ba8b-372a43bbed26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:32.116314 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a0ccb-5a6b-493a-ba8b-372a43bbed26 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:32.116594 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a0ccb-5a6b-493a-ba8b-372a43bbed26 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:32.116868 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a0ccb-5a6b-493a-ba8b-372a43bbed26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:32.130164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a0ccb-5a6b-493a-ba8b-372a43bbed26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:32.137857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-547a0ccb-5a6b-493a-ba8b-372a43bbed26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:32.137857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:32.137857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:32.137857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:32.137857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:32.137857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:32.137857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:32.137857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:32.137857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:32.137857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:32.137857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:32.137857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:32.137857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:32.137857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:32.137857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:32.137857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:32.138755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:32.138755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:32.138755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:32.138755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:32.138755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:32.138755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:32.138755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:32.138755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:32.138755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:32.138755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:32.138755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:32.138755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:32.138755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:32.138755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:32.138755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:32.138755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:32.139570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:32.139570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:32.139570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:32.139570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:32.139570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:32.139570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:32.139570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:32.139570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:32.139570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:32.139570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:32.139570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:32.139570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:32.139570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 277/546] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:32:32.149936 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a0ccb-5a6b-493a-ba8b-372a43bbed26 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:32.202467 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283791a-bf4f-43a6-91fd-f79a01bfed7f None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:32.256111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742de0e1-7890-4ebf-b0db-6a0fb7284a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:32.256934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742de0e1-7890-4ebf-b0db-6a0fb7284a40 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:32.257220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742de0e1-7890-4ebf-b0db-6a0fb7284a40 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:32.257500 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742de0e1-7890-4ebf-b0db-6a0fb7284a40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:32.275908 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742de0e1-7890-4ebf-b0db-6a0fb7284a40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:32.284681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-742de0e1-7890-4ebf-b0db-6a0fb7284a40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:32.284681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:32.284681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:32.284681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:32.284681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:32.284681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:32.284681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:32.284681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:32.284681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:32.284681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:32.284681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:32.284681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:32.284681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:32.284681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:32.284681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:32.284681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:32.286113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:32.286113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:32.286113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:32.286113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:32.286113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:32.286113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:32.286113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:32.286113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:32.286113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:32.286113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:32.286113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:32.286113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:32.286113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:32.286113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:32.286113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:32.286113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:32.286921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:32.286921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:32.286921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:32.286921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:32.286921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:32.286921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:32.286921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:32.286921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:32.286921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:32.286921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:32.286921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:32.286921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:32.291372 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 270/547] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:32.299575 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742de0e1-7890-4ebf-b0db-6a0fb7284a40 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:32.311737 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:32.339533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a0ccb-5a6b-493a-ba8b-372a43bbed26 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:32.355405 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:32.390665 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec40b48-b0e8-4575-9138-20580af3e1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:32.391342 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec40b48-b0e8-4575-9138-20580af3e1c3 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:32.391646 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec40b48-b0e8-4575-9138-20580af3e1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:32.391921 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec40b48-b0e8-4575-9138-20580af3e1c3 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:32.392678 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec40b48-b0e8-4575-9138-20580af3e1c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:32.392984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec40b48-b0e8-4575-9138-20580af3e1c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:32.394430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec40b48-b0e8-4575-9138-20580af3e1c3 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:32.416933 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 278/548] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:32:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:32.418011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec40b48-b0e8-4575-9138-20580af3e1c3 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:32.486144 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742de0e1-7890-4ebf-b0db-6a0fb7284a40 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:32.569179 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec40b48-b0e8-4575-9138-20580af3e1c3 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:32.575474 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff2734-063a-4414-af76-37ef6a2443b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:32.576190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff2734-063a-4414-af76-37ef6a2443b5 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:32.576526 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff2734-063a-4414-af76-37ef6a2443b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:32.576812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff2734-063a-4414-af76-37ef6a2443b5 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:32.577653 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff2734-063a-4414-af76-37ef6a2443b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:32.577875 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff2734-063a-4414-af76-37ef6a2443b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:32.579287 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff2734-063a-4414-af76-37ef6a2443b5 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:32.598821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 271/549] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:32:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:32.610563 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff2734-063a-4414-af76-37ef6a2443b5 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:32.615401 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f1f3d-c56b-4525-9de7-751cf5ee4d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:32.616672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f1f3d-c56b-4525-9de7-751cf5ee4d97 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:32.617142 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f1f3d-c56b-4525-9de7-751cf5ee4d97 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:32.617425 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f1f3d-c56b-4525-9de7-751cf5ee4d97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:32.629343 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f1f3d-c56b-4525-9de7-751cf5ee4d97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:32.630871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f1f3d-c56b-4525-9de7-751cf5ee4d97 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:32.632763 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:32.670248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-5d6f1f3d-c56b-4525-9de7-751cf5ee4d97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:32.670248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:32.670248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:32.670248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:32.670248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:32.670248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:32.670248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:32.670248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:32.670248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:32.670248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:32.670248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:32.670248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:32.670248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:32.670248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:32.670248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:32.671327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:32.671327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:32.671327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:32.671327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:32.671327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:32.671327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:32.671327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:32.671327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:32.671327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:32.671327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:32.671327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:32.671327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:32.671327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:32.671327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:32.671327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:32.671327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:32.671327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:32.672222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:32.672222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:32.672222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:32.672222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:32.674027 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 279/550] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:32.680079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f1f3d-c56b-4525-9de7-751cf5ee4d97 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:32.803712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:32.823143 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f1f3d-c56b-4525-9de7-751cf5ee4d97 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:32.833281 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff2734-063a-4414-af76-37ef6a2443b5 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:32.860778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d3499-12ff-43ec-91cd-199476718055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:32.861557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d3499-12ff-43ec-91cd-199476718055 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:32.861843 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d3499-12ff-43ec-91cd-199476718055 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:32.862197 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d3499-12ff-43ec-91cd-199476718055 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:32.863093 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d3499-12ff-43ec-91cd-199476718055 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:32.863799 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d3499-12ff-43ec-91cd-199476718055 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:32.865705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d3499-12ff-43ec-91cd-199476718055 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:32.885893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 280/551] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:32.895775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d3499-12ff-43ec-91cd-199476718055 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:32.939662 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bfdaa-dac6-4f8f-bbc0-cee029bfb474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:32.939662 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bfdaa-dac6-4f8f-bbc0-cee029bfb474 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:32.939662 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bfdaa-dac6-4f8f-bbc0-cee029bfb474 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:32.939662 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bfdaa-dac6-4f8f-bbc0-cee029bfb474 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:32.958099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1bfdaa-dac6-4f8f-bbc0-cee029bfb474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:32.960988 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1bfdaa-dac6-4f8f-bbc0-cee029bfb474 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:32.995513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-4c1bfdaa-dac6-4f8f-bbc0-cee029bfb474 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:32.995513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:32.995513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:32.995513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:32.995513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:32.995513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:32.995513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:32.995513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:32.995513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:32.995513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:32.995513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:32.995513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:32.995513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:32.995513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:32.995513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:32.996515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:32.996515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:32.996515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:32.996515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:32.996515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:32.996515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:32.996515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:32.996515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:32.996515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:32.996515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:32.996515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:32.996515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:32.996515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:32.996515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:32.996515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:32.996515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:32.996515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:32.997469 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:32.997469 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:32.997469 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:32.997469 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:33.005140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 272/552] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:33.006266 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bfdaa-dac6-4f8f-bbc0-cee029bfb474 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:33.022739 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:33.117142 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d3499-12ff-43ec-91cd-199476718055 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:33.164941 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a425d-a536-4ed1-acdf-0714e446899b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:33.166082 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a425d-a536-4ed1-acdf-0714e446899b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:33.166564 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a425d-a536-4ed1-acdf-0714e446899b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:33.167056 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a425d-a536-4ed1-acdf-0714e446899b None admin] PATH_INFO: `/v3/users/nova` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:33.175006 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a425d-a536-4ed1-acdf-0714e446899b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:33.176947 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a425d-a536-4ed1-acdf-0714e446899b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:33.193949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-8e1a425d-a536-4ed1-acdf-0714e446899b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 22 11:32:33.193949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:33.193949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:33.193949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:33.193949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:33.193949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:33.193949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:33.193949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:33.193949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:33.193949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:33.193949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:33.193949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:33.193949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:33.193949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:33.193949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:33.193949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:33.194913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:33.194913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:33.194913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:33.194913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:33.194913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:33.194913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:33.194913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:33.194913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:33.194913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:33.194913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:33.194913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:33.194913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:33.194913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:33.194913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:33.194913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:33.194913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:33.194913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:33.194913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:33.196261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:33.196261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:33.196261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:33.196261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:33.196261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:33.196261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:33.196261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:33.196261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:33.196261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 22 11:32:33.196261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:33.197085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bfdaa-dac6-4f8f-bbc0-cee029bfb474 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:33.198973 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 281/553] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jun 22 11:32:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:33.200785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d01f05-3254-450b-b0a3-04c1492dab51 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:33.201324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d01f05-3254-450b-b0a3-04c1492dab51 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:33.201800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d01f05-3254-450b-b0a3-04c1492dab51 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:33.208521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 282/554] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:33.209868 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d01f05-3254-450b-b0a3-04c1492dab51 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:33.244139 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e7188-1d1e-4e8f-9359-f8dab27cd4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:33.244809 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e7188-1d1e-4e8f-9359-f8dab27cd4d4 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:33.245092 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e7188-1d1e-4e8f-9359-f8dab27cd4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:33.245367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e7188-1d1e-4e8f-9359-f8dab27cd4d4 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:33.246081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e7188-1d1e-4e8f-9359-f8dab27cd4d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:33.246386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e7188-1d1e-4e8f-9359-f8dab27cd4d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:33.247788 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e7188-1d1e-4e8f-9359-f8dab27cd4d4 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:33.271237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 273/555] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:33.271237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2a703-2eaa-4460-9360-444ec8b86f19 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:33.271237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2a703-2eaa-4460-9360-444ec8b86f19 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:33.271237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2a703-2eaa-4460-9360-444ec8b86f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:33.275569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:33.291012 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-03d2a703-2eaa-4460-9360-444ec8b86f19 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:33.292867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-03d2a703-2eaa-4460-9360-444ec8b86f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:33.340543 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d01f05-3254-450b-b0a3-04c1492dab51 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:33.359789 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:33.384758 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002d441-279a-4f7d-a784-bacc9e4ac79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:33.385430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002d441-279a-4f7d-a784-bacc9e4ac79a None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:33.385706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002d441-279a-4f7d-a784-bacc9e4ac79a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:33.386058 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002d441-279a-4f7d-a784-bacc9e4ac79a None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:33.386999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0002d441-279a-4f7d-a784-bacc9e4ac79a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:33.387312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0002d441-279a-4f7d-a784-bacc9e4ac79a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:33.388779 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0002d441-279a-4f7d-a784-bacc9e4ac79a None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:33.389014 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 274/556] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:33.390295 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2a703-2eaa-4460-9360-444ec8b86f19 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:33.427316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 283/557] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jun 22 11:32:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:33.428268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002d441-279a-4f7d-a784-bacc9e4ac79a None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:33.498318 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2a703-2eaa-4460-9360-444ec8b86f19 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:33.533889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:33.535882 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73958aa7-8fb6-4f3d-864c-d05fa75fd599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:33.536882 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73958aa7-8fb6-4f3d-864c-d05fa75fd599 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:33.538381 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73958aa7-8fb6-4f3d-864c-d05fa75fd599 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:33.538381 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73958aa7-8fb6-4f3d-864c-d05fa75fd599 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:33.538659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002d441-279a-4f7d-a784-bacc9e4ac79a None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:33.546792 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73958aa7-8fb6-4f3d-864c-d05fa75fd599 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:33.548835 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73958aa7-8fb6-4f3d-864c-d05fa75fd599 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:33.570803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-73958aa7-8fb6-4f3d-864c-d05fa75fd599 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 22 11:32:33.570803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:33.570803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:33.570803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:33.570803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:33.570803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:33.570803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:33.570803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:33.570803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:33.570803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:33.570803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:33.570803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:33.570803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:33.570803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:33.570803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:33.570803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:33.572461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:33.572461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:33.572461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:33.572461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:33.572461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:33.572461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:33.572461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:33.572461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:33.572461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:33.572461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:33.572461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:33.572461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:33.572461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:33.572461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:33.572461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:33.572461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:33.572461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:33.572461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:33.573412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:33.573412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:33.573412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:33.573412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:33.573412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:33.573412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:33.573412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:33.573412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:33.573412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 22 11:32:33.573412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:33.579385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 275/558] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:32:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:33.579385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73958aa7-8fb6-4f3d-864c-d05fa75fd599 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:33.585086 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bfa52-2fab-4717-9b46-e9706554cea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:33.586021 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bfa52-2fab-4717-9b46-e9706554cea8 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:33.586976 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bfa52-2fab-4717-9b46-e9706554cea8 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:33.587378 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bfa52-2fab-4717-9b46-e9706554cea8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:33.603208 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1bfa52-2fab-4717-9b46-e9706554cea8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:33.614214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-cb1bfa52-2fab-4717-9b46-e9706554cea8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:33.614214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:33.614214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:33.614214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:33.614214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:33.614214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:33.614214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:33.614214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:33.614214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:33.614214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:33.614214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:33.614214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:33.614214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:33.614214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:33.614214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:33.614214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:33.616254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:33.616254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:33.616254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:33.616254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:33.616254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:33.616254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:33.616254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:33.616254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:33.616254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:33.616254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:33.616254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:33.616254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:33.616254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:33.616254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:33.616254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:33.616254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:33.617733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:33.617733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:33.617733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:33.617733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:33.617733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:33.617733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:33.617733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:33.617733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:33.617733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:33.617733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:33.617733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:33.617733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:33.623390 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 284/559] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:33.623390 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bfa52-2fab-4717-9b46-e9706554cea8 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:33.664879 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:33.702084 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73958aa7-8fb6-4f3d-864c-d05fa75fd599 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:33.744721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bfa52-2fab-4717-9b46-e9706554cea8 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:33.750924 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add61da2-3786-4b8c-9767-e9704d7688a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:33.751838 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add61da2-3786-4b8c-9767-e9704d7688a7 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:33.752227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add61da2-3786-4b8c-9767-e9704d7688a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:33.752607 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add61da2-3786-4b8c-9767-e9704d7688a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:33.763781 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add61da2-3786-4b8c-9767-e9704d7688a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:33.765658 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add61da2-3786-4b8c-9767-e9704d7688a7 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:33.786965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3533d-fca9-4711-8efd-2536812244f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:33.787945 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3533d-fca9-4711-8efd-2536812244f6 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:33.788370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3533d-fca9-4711-8efd-2536812244f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:33.788787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b3533d-fca9-4711-8efd-2536812244f6 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:33.789032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-add61da2-3786-4b8c-9767-e9704d7688a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:33.789032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:33.789032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:33.789032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:33.789032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:33.789032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:33.789032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:33.789032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:33.789032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:33.789032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:33.789032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:33.789032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:33.789032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:33.789032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:33.789032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:33.789938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:33.789938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:33.789938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:33.789938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:33.789938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:33.789938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:33.789938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:33.789938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:33.789938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:33.789938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:33.789938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:33.789938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:33.789938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:33.789938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:33.789938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:33.789938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:33.789938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:33.790799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:33.790799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:33.790799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:33.790799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:33.792286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3533d-fca9-4711-8efd-2536812244f6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:33.792782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3533d-fca9-4711-8efd-2536812244f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:33.794515 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b3533d-fca9-4711-8efd-2536812244f6 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:33.795786 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 276/560] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:33.796810 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add61da2-3786-4b8c-9767-e9704d7688a7 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:33.825407 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 285/561] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jun 22 11:32:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:33.825407 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3533d-fca9-4711-8efd-2536812244f6 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:33.852848 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:33.919204 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add61da2-3786-4b8c-9767-e9704d7688a7 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:33.952645 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:33.964303 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b3533d-fca9-4711-8efd-2536812244f6 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:33.973167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2b9c6-2e41-4bf7-8c7f-8aba546326e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:33.974245 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2b9c6-2e41-4bf7-8c7f-8aba546326e5 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:33.974786 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2b9c6-2e41-4bf7-8c7f-8aba546326e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:33.975323 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2b9c6-2e41-4bf7-8c7f-8aba546326e5 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:33.976450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2b9c6-2e41-4bf7-8c7f-8aba546326e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:33.976976 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2b9c6-2e41-4bf7-8c7f-8aba546326e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:33.978576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2b9c6-2e41-4bf7-8c7f-8aba546326e5 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:33.988945 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 277/562] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:32:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:33.990178 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2b9c6-2e41-4bf7-8c7f-8aba546326e5 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:34.007794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c3681-c751-4bb8-8f6f-c8bae9c4dbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:34.008781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c3681-c751-4bb8-8f6f-c8bae9c4dbda None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:34.009268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c3681-c751-4bb8-8f6f-c8bae9c4dbda None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:34.009761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5c3681-c751-4bb8-8f6f-c8bae9c4dbda None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:34.010958 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c3681-c751-4bb8-8f6f-c8bae9c4dbda None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:34.011624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c3681-c751-4bb8-8f6f-c8bae9c4dbda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:34.013259 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5c3681-c751-4bb8-8f6f-c8bae9c4dbda None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:34.033360 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 286/563] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:34.035085 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c3681-c751-4bb8-8f6f-c8bae9c4dbda None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:34.111279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2b9c6-2e41-4bf7-8c7f-8aba546326e5 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:34.118444 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:34.145469 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5c3681-c751-4bb8-8f6f-c8bae9c4dbda None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:34.153553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf7cea-fcf0-4302-89d2-b0461eeccd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:34.154208 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf7cea-fcf0-4302-89d2-b0461eeccd44 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:34.154867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf7cea-fcf0-4302-89d2-b0461eeccd44 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:34.155155 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf7cea-fcf0-4302-89d2-b0461eeccd44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:34.179337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6fa90-6ded-4982-b80c-76d096c8b851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:34.180149 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6fa90-6ded-4982-b80c-76d096c8b851 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:34.180430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6fa90-6ded-4982-b80c-76d096c8b851 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:34.180699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6fa90-6ded-4982-b80c-76d096c8b851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:34.190811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6fa90-6ded-4982-b80c-76d096c8b851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:34.192290 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6fa90-6ded-4982-b80c-76d096c8b851 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:34.209359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-77f6fa90-6ded-4982-b80c-76d096c8b851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:34.209359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:34.209359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:34.209359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:34.209359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:34.209359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:34.209359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:34.209359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:34.209359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:34.209359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:34.209359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:34.209359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:34.209359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:34.209359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:34.209359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:34.210313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:34.210313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:34.210313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:34.210313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:34.210313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:34.210313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:34.210313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:34.210313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:34.210313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:34.210313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:34.210313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:34.210313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:34.210313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:34.210313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:34.210313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:34.210313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:34.210313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:34.211220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:34.211220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:34.211220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:34.211220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:34.211220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 287/564] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:34.212248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6fa90-6ded-4982-b80c-76d096c8b851 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:34.215488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:34.227804 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf7cea-fcf0-4302-89d2-b0461eeccd44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:34.229202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf7cea-fcf0-4302-89d2-b0461eeccd44 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:34.315247 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6fa90-6ded-4982-b80c-76d096c8b851 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:34.338536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 278/565] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:34.339894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56701f7d-e637-408a-92f6-9176833b7bd6 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:34.340182 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56701f7d-e637-408a-92f6-9176833b7bd6 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:34.340459 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56701f7d-e637-408a-92f6-9176833b7bd6 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:34.341715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 279/566] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:34.342759 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19dc3e-9e36-4cbe-acb1-0f3264d36312 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:34.343044 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19dc3e-9e36-4cbe-acb1-0f3264d36312 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:34.343318 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19dc3e-9e36-4cbe-acb1-0f3264d36312 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:34.344566 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 280/567] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:34.345250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19dc3e-9e36-4cbe-acb1-0f3264d36312 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:34.355588 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:34.357540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fc282-db30-4b32-be51-0ea3a391c275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:34.358335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fc282-db30-4b32-be51-0ea3a391c275 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:34.358619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fc282-db30-4b32-be51-0ea3a391c275 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:34.358894 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fc282-db30-4b32-be51-0ea3a391c275 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:34.365688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524fc282-db30-4b32-be51-0ea3a391c275 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:34.367125 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524fc282-db30-4b32-be51-0ea3a391c275 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:34.381900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-524fc282-db30-4b32-be51-0ea3a391c275 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:34.381900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:34.381900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:34.381900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:34.381900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:34.381900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:34.381900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:34.381900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:34.381900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:34.381900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:34.381900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:34.381900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:34.381900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:34.381900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:34.381900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:34.382810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:34.382810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:34.382810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:34.382810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:34.382810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:34.382810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:34.382810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:34.382810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:34.382810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:34.382810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:34.382810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:34.382810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:34.382810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:34.382810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:34.382810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:34.382810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:34.382810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:34.383755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:34.383755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:34.383755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:34.383755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:34.383755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:34.383755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:34.383755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:34.383755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:34.383755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 288/568] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:34.384241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fc282-db30-4b32-be51-0ea3a391c275 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:34.448359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:34.451520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19dc3e-9e36-4cbe-acb1-0f3264d36312 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:34.486155 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0e153-4dad-4189-8872-7f656323eb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:34.486815 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0e153-4dad-4189-8872-7f656323eb37 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:34.486940 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fc282-db30-4b32-be51-0ea3a391c275 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:34.487095 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0e153-4dad-4189-8872-7f656323eb37 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:34.487382 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0e153-4dad-4189-8872-7f656323eb37 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:34.488131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0e153-4dad-4189-8872-7f656323eb37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:34.488437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0e153-4dad-4189-8872-7f656323eb37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:34.489825 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0e153-4dad-4189-8872-7f656323eb37 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:34.500812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 281/569] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:34.502174 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752d31d-0b98-4645-9c15-837b5a3e0a5b None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:34.502465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752d31d-0b98-4645-9c15-837b5a3e0a5b None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:34.502742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752d31d-0b98-4645-9c15-837b5a3e0a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:34.520534 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-1752d31d-0b98-4645-9c15-837b5a3e0a5b None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:34.521743 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-1752d31d-0b98-4645-9c15-837b5a3e0a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:34.521992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85a543-df46-46e9-a5d3-628264a168cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:34.522789 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85a543-df46-46e9-a5d3-628264a168cb None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:34.523072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85a543-df46-46e9-a5d3-628264a168cb None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:34.523368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85a543-df46-46e9-a5d3-628264a168cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:34.536616 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85a543-df46-46e9-a5d3-628264a168cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:34.538166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85a543-df46-46e9-a5d3-628264a168cb None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:34.556345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-ac85a543-df46-46e9-a5d3-628264a168cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:34.556345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:34.556345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:34.556345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:34.556345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:34.556345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:34.556345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:34.556345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:34.556345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:34.556345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:34.556345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:34.556345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:34.556345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:34.556345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:34.556345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:34.557419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:34.557419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:34.557419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:34.557419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:34.557419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:34.557419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:34.557419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:34.557419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:34.557419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:34.557419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:34.557419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:34.557419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:34.557419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:34.557419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:34.557419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:34.557419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:34.557419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:34.558588 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:34.558588 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:34.558588 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:34.558588 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:34.558588 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 289/570] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:34.559048 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad4fec-99e9-44ba-bc51-b57a5ef737b2 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:34.559334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad4fec-99e9-44ba-bc51-b57a5ef737b2 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:34.559634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad4fec-99e9-44ba-bc51-b57a5ef737b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:34.575594 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-b2ad4fec-99e9-44ba-bc51-b57a5ef737b2 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:34.576718 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b2ad4fec-99e9-44ba-bc51-b57a5ef737b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:34.586026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:34.593919 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 282/571] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:34.594721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752d31d-0b98-4645-9c15-837b5a3e0a5b None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:34.658291 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 290/572] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:34.659253 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad4fec-99e9-44ba-bc51-b57a5ef737b2 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:34.696637 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752d31d-0b98-4645-9c15-837b5a3e0a5b None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:34.711481 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:34.732325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747be656-37d3-400c-92e8-a3e88560ea2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:34.732980 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747be656-37d3-400c-92e8-a3e88560ea2e None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:34.733257 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747be656-37d3-400c-92e8-a3e88560ea2e None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:34.733532 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747be656-37d3-400c-92e8-a3e88560ea2e None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:34.734309 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747be656-37d3-400c-92e8-a3e88560ea2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:34.734615 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747be656-37d3-400c-92e8-a3e88560ea2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:34.736082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747be656-37d3-400c-92e8-a3e88560ea2e None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:34.747577 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 283/573] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:34.748468 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747be656-37d3-400c-92e8-a3e88560ea2e None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:34.770861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad4fec-99e9-44ba-bc51-b57a5ef737b2 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:34.804239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76d095-d477-441d-8741-992f72f9dd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:34.805024 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76d095-d477-441d-8741-992f72f9dd5f None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:34.805308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76d095-d477-441d-8741-992f72f9dd5f None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:34.805584 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76d095-d477-441d-8741-992f72f9dd5f None admin] PATH_INFO: `/v3/users/glance` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:34.805860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:34.813004 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76d095-d477-441d-8741-992f72f9dd5f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:34.814543 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76d095-d477-441d-8741-992f72f9dd5f None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:34.828776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-0b76d095-d477-441d-8741-992f72f9dd5f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 22 11:32:34.828776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:34.828776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:34.828776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:34.828776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:34.828776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:34.828776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:34.828776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:34.828776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:34.828776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:34.828776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:34.828776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:34.828776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:34.828776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:34.828776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:34.828776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:34.829720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:34.829720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:34.829720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:34.829720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:34.829720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:34.829720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:34.829720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:34.829720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:34.829720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:34.829720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:34.829720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:34.829720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:34.829720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:34.829720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:34.829720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:34.829720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:34.829720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:34.829720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:34.830644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:34.830644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:34.830644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:34.830644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:34.830644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:34.830644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:34.830644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:34.830644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:34.830644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 22 11:32:34.830644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:34.830644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 291/574] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:34.830644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76d095-d477-441d-8741-992f72f9dd5f None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:34.854108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747be656-37d3-400c-92e8-a3e88560ea2e None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:34.887913 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a94cc-875c-432e-b618-29f7db3e70de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:34.888573 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a94cc-875c-432e-b618-29f7db3e70de None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:34.888856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a94cc-875c-432e-b618-29f7db3e70de None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:34.889136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a94cc-875c-432e-b618-29f7db3e70de None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:34.889842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a94cc-875c-432e-b618-29f7db3e70de None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:34.890442 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a94cc-875c-432e-b618-29f7db3e70de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:34.891814 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a94cc-875c-432e-b618-29f7db3e70de None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:34.908595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 284/575] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:34.908595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a94cc-875c-432e-b618-29f7db3e70de None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:34.935302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76d095-d477-441d-8741-992f72f9dd5f None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:34.937650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:34.974677 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211bed9-7e90-4491-b040-ba7a60cd3e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:34.975499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211bed9-7e90-4491-b040-ba7a60cd3e9e None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:34.975778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211bed9-7e90-4491-b040-ba7a60cd3e9e None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:34.976045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211bed9-7e90-4491-b040-ba7a60cd3e9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:34.986346 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211bed9-7e90-4491-b040-ba7a60cd3e9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:34.994483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-5211bed9-7e90-4491-b040-ba7a60cd3e9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:34.994483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:34.994483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:34.994483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:34.994483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:34.994483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:34.994483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:34.994483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:34.994483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:34.994483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:34.994483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:34.994483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:34.994483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:34.994483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:34.994483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:34.994483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:34.995496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:34.995496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:34.995496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:34.995496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:34.995496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:34.995496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:34.995496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:34.995496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:34.995496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:34.995496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:34.995496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:34.995496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:34.995496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:34.995496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:34.995496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:34.995496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:34.996357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:34.996357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:34.996357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:34.996357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:34.996357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:34.996357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:34.996357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:34.996357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:34.996357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:34.996357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:34.996357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:34.996357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:34.996357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 292/576] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:34.997072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211bed9-7e90-4491-b040-ba7a60cd3e9e None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:35.020770 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a94cc-875c-432e-b618-29f7db3e70de None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:35.038074 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:35.057919 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26b93e-16ce-4c3d-a3bb-0f5928e89ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:35.058717 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26b93e-16ce-4c3d-a3bb-0f5928e89ee8 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:35.059013 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26b93e-16ce-4c3d-a3bb-0f5928e89ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:35.059290 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26b93e-16ce-4c3d-a3bb-0f5928e89ee8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:35.070219 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26b93e-16ce-4c3d-a3bb-0f5928e89ee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:35.077817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-9d26b93e-16ce-4c3d-a3bb-0f5928e89ee8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:35.077817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:35.077817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:35.077817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:35.077817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:35.077817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:35.077817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:35.077817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:35.077817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:35.077817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:35.077817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:35.077817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:35.077817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:35.077817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:35.077817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:35.077817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:35.079026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:35.079026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:35.079026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:35.079026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:35.079026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:35.079026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:35.079026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:35.079026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:35.079026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:35.079026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:35.079026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:35.079026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:35.079026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:35.079026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:35.079026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:35.079026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:35.080127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:35.080127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:35.080127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:35.080127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:35.080127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:35.080127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:35.080127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:35.080127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:35.080127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:35.080127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:35.080127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:35.080127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:35.083496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 285/577] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:35.084803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26b93e-16ce-4c3d-a3bb-0f5928e89ee8 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:35.105614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211bed9-7e90-4491-b040-ba7a60cd3e9e None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:35.140871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039a3d6-73ce-4f25-8f7b-193f700c5cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:35.141791 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039a3d6-73ce-4f25-8f7b-193f700c5cb0 None admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:35.142068 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039a3d6-73ce-4f25-8f7b-193f700c5cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:35.142339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039a3d6-73ce-4f25-8f7b-193f700c5cb0 None admin] PATH_INFO: `/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/479a376dcac14c509c1c55f0046d88cf/roles/64c2d69f15ca458c8267c1acfd42d9fc` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:35.159447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039a3d6-73ce-4f25-8f7b-193f700c5cb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=47d2bb094c9843c9a7726f1d672d2433, user_id=479a376dcac14c509c1c55f0046d88cf, role_id=64c2d69f15ca458c8267c1acfd42d9fc)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:35.161712 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4039a3d6-73ce-4f25-8f7b-193f700c5cb0 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:35.177660 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 293/578] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:32:34 2022] PUT /identity/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/479a376dcac14c509c1c55f0046d88cf/roles/64c2d69f15ca458c8267c1acfd42d9fc => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:32:35.178559 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039a3d6-73ce-4f25-8f7b-193f700c5cb0 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:35.188206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26b93e-16ce-4c3d-a3bb-0f5928e89ee8 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:35.195269 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:35.229655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb24dc-a2fb-4b49-9f70-a6f4f2e14c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:35.230354 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb24dc-a2fb-4b49-9f70-a6f4f2e14c83 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:35.230639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb24dc-a2fb-4b49-9f70-a6f4f2e14c83 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:35.230928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb24dc-a2fb-4b49-9f70-a6f4f2e14c83 None admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:35.231839 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb24dc-a2fb-4b49-9f70-a6f4f2e14c83 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:35.232209 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb24dc-a2fb-4b49-9f70-a6f4f2e14c83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:35.233729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb24dc-a2fb-4b49-9f70-a6f4f2e14c83 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:35.256563 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 286/579] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jun 22 11:32:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:35.257456 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb24dc-a2fb-4b49-9f70-a6f4f2e14c83 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:35.297617 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:35.302039 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039a3d6-73ce-4f25-8f7b-193f700c5cb0 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:35.344705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e2bf6-e058-42f3-82f5-4ca05ca97ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:35.345436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e2bf6-e058-42f3-82f5-4ca05ca97ecd None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:35.345712 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e2bf6-e058-42f3-82f5-4ca05ca97ecd None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:35.345981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e2bf6-e058-42f3-82f5-4ca05ca97ecd None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:35.354018 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e2bf6-e058-42f3-82f5-4ca05ca97ecd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:35.355625 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e2bf6-e058-42f3-82f5-4ca05ca97ecd None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:35.371262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-f55e2bf6-e058-42f3-82f5-4ca05ca97ecd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:35.371262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:35.371262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:35.371262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:35.371262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:35.371262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:35.371262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:35.371262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:35.371262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:35.371262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:35.371262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:35.371262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:35.371262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:35.371262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:35.371262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:35.372222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:35.372222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:35.372222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:35.372222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:35.372222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:35.372222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:35.372222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:35.372222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:35.372222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:35.372222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:35.372222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:35.372222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:35.372222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:35.372222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:35.372222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:35.372222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:35.372222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:35.373114 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:35.373114 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:35.373114 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:35.373114 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:35.373114 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:35.373114 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:35.373114 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:35.373114 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:35.373114 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 294/580] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:35.373600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e2bf6-e058-42f3-82f5-4ca05ca97ecd None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:35.383975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb24dc-a2fb-4b49-9f70-a6f4f2e14c83 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:35.420026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff64eb-547d-4cd6-9b7f-c4baee673529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:35.420697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff64eb-547d-4cd6-9b7f-c4baee673529 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:35.420979 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff64eb-547d-4cd6-9b7f-c4baee673529 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:35.421253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff64eb-547d-4cd6-9b7f-c4baee673529 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:35.422008 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff64eb-547d-4cd6-9b7f-c4baee673529 None admin] RBAC: Adding query filter params (name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:35.422312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff64eb-547d-4cd6-9b7f-c4baee673529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:35.423706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff64eb-547d-4cd6-9b7f-c4baee673529 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:35.432423 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 287/581] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:32:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:35.433093 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff64eb-547d-4cd6-9b7f-c4baee673529 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:35.463247 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:35.490810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e2bf6-e058-42f3-82f5-4ca05ca97ecd None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:35.527744 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ac45f-94e9-4ebd-bdb7-299b05c6ded8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:35.528432 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ac45f-94e9-4ebd-bdb7-299b05c6ded8 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:35.528711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ac45f-94e9-4ebd-bdb7-299b05c6ded8 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:35.528988 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ac45f-94e9-4ebd-bdb7-299b05c6ded8 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:35.529744 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ac45f-94e9-4ebd-bdb7-299b05c6ded8 None admin] RBAC: Adding query filter params (name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:35.530053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ac45f-94e9-4ebd-bdb7-299b05c6ded8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:35.531454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ac45f-94e9-4ebd-bdb7-299b05c6ded8 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:35.537198 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:35.544448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 295/582] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:32:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:35.544448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ac45f-94e9-4ebd-bdb7-299b05c6ded8 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:35.551403 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff64eb-547d-4cd6-9b7f-c4baee673529 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:35.590015 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0f186-7a35-4ed2-9008-079b20e2c086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:35.591157 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0f186-7a35-4ed2-9008-079b20e2c086 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:35.591631 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0f186-7a35-4ed2-9008-079b20e2c086 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:35.591906 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0f186-7a35-4ed2-9008-079b20e2c086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:35.603684 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0f186-7a35-4ed2-9008-079b20e2c086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:35.605185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0f186-7a35-4ed2-9008-079b20e2c086 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:35.627099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-92d0f186-7a35-4ed2-9008-079b20e2c086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:35.627099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:35.627099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:35.627099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:35.627099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:35.627099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:35.627099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:35.627099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:35.627099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:35.627099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:35.627099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:35.627099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:35.627099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:35.627099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:35.627099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:35.628241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:35.628241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:35.628241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:35.628241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:35.628241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:35.628241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:35.628241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:35.628241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:35.628241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:35.628241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:35.628241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:35.628241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:35.628241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:35.628241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:35.628241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:35.628241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:35.628241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:35.634228 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:35.634228 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:35.634228 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:35.634228 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:35.634228 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 288/583] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:35.634228 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0f186-7a35-4ed2-9008-079b20e2c086 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:35.661530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ac45f-94e9-4ebd-bdb7-299b05c6ded8 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:35.696381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5abc18e-8ea0-4c66-be4a-1959b9090d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:35.697038 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5abc18e-8ea0-4c66-be4a-1959b9090d59 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:35.697318 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5abc18e-8ea0-4c66-be4a-1959b9090d59 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:35.697589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5abc18e-8ea0-4c66-be4a-1959b9090d59 None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:35.698375 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5abc18e-8ea0-4c66-be4a-1959b9090d59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:35.698679 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5abc18e-8ea0-4c66-be4a-1959b9090d59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:35.700129 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5abc18e-8ea0-4c66-be4a-1959b9090d59 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:35.702644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:35.717518 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 296/584] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:35.717518 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5abc18e-8ea0-4c66-be4a-1959b9090d59 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:35.745244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0f186-7a35-4ed2-9008-079b20e2c086 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:35.782070 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1119e4-1b48-4e92-b42e-827895144d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:35.782885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1119e4-1b48-4e92-b42e-827895144d87 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:35.783189 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1119e4-1b48-4e92-b42e-827895144d87 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:35.783514 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1119e4-1b48-4e92-b42e-827895144d87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:35.794399 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1119e4-1b48-4e92-b42e-827895144d87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:35.795916 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1119e4-1b48-4e92-b42e-827895144d87 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:35.805071 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:35.815714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-1f1119e4-1b48-4e92-b42e-827895144d87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:35.815714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:35.815714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:35.815714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:35.815714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:35.815714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:35.815714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:35.815714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:35.815714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:35.815714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:35.815714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:35.815714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:35.815714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:35.815714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:35.815714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:35.817616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:35.817616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:35.817616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:35.817616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:35.817616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:35.817616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:35.817616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:35.817616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:35.817616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:35.817616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:35.817616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:35.817616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:35.817616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:35.817616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:35.817616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:35.817616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:35.817616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:35.818830 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:35.818830 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:35.818830 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:35.818830 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:35.825914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 289/585] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:35.826861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1119e4-1b48-4e92-b42e-827895144d87 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:35.829632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5abc18e-8ea0-4c66-be4a-1959b9090d59 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:35.874899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947befa-224f-4a3d-8a5e-ff1066877c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:35.875995 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947befa-224f-4a3d-8a5e-ff1066877c80 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:35.876411 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947befa-224f-4a3d-8a5e-ff1066877c80 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:35.876826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947befa-224f-4a3d-8a5e-ff1066877c80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:35.890945 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947befa-224f-4a3d-8a5e-ff1066877c80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:35.892756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947befa-224f-4a3d-8a5e-ff1066877c80 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:35.913565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-b947befa-224f-4a3d-8a5e-ff1066877c80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:35.913565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:35.913565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:35.913565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:35.913565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:35.913565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:35.913565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:35.913565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:35.913565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:35.913565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:35.913565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:35.913565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:35.913565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:35.913565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:35.913565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:35.914578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:35.914578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:35.914578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:35.914578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:35.914578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:35.914578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:35.914578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:35.914578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:35.914578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:35.914578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:35.914578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:35.914578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:35.914578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:35.914578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:35.914578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:35.914578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:35.914578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:35.915982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:35.915982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:35.915982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:35.915982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:35.915982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 297/586] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:35.916295 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947befa-224f-4a3d-8a5e-ff1066877c80 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:35.950030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1119e4-1b48-4e92-b42e-827895144d87 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:35.977783 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:35.992286 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c4932-26f3-4ac6-94a2-d173dc756b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:35.992964 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c4932-26f3-4ac6-94a2-d173dc756b39 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:35.993252 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c4932-26f3-4ac6-94a2-d173dc756b39 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:35.993534 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c4932-26f3-4ac6-94a2-d173dc756b39 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:35.994260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c4932-26f3-4ac6-94a2-d173dc756b39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:35.994573 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c4932-26f3-4ac6-94a2-d173dc756b39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:35.995995 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c4932-26f3-4ac6-94a2-d173dc756b39 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:36.012697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 290/587] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:36.012697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c4932-26f3-4ac6-94a2-d173dc756b39 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:36.038437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947befa-224f-4a3d-8a5e-ff1066877c80 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:36.072206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:36.073916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017e01b-755f-4c72-a860-f2b13ee1c29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:36.074618 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017e01b-755f-4c72-a860-f2b13ee1c29e None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:36.074945 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017e01b-755f-4c72-a860-f2b13ee1c29e None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:36.075254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017e01b-755f-4c72-a860-f2b13ee1c29e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:36.076132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017e01b-755f-4c72-a860-f2b13ee1c29e None admin] RBAC: Adding query filter params (role.id=64c2d69f15ca458c8267c1acfd42d9fc, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=dcd642e290e44e67b56eb3b8361d7af3) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:36.076443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017e01b-755f-4c72-a860-f2b13ee1c29e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:36.078008 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017e01b-755f-4c72-a860-f2b13ee1c29e None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:36.084689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 298/588] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:32:35 2022] GET /identity/v3/role_assignments?user.id=dcd642e290e44e67b56eb3b8361d7af3&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=64c2d69f15ca458c8267c1acfd42d9fc => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:36.085311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017e01b-755f-4c72-a860-f2b13ee1c29e None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:36.140250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c4932-26f3-4ac6-94a2-d173dc756b39 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:36.179940 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6448d-a6b9-4b82-8e93-e70d2bf62979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:36.180615 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6448d-a6b9-4b82-8e93-e70d2bf62979 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:36.180898 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6448d-a6b9-4b82-8e93-e70d2bf62979 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:36.181174 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6448d-a6b9-4b82-8e93-e70d2bf62979 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:36.181967 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6448d-a6b9-4b82-8e93-e70d2bf62979 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:36.182278 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6448d-a6b9-4b82-8e93-e70d2bf62979 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:36.183860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6448d-a6b9-4b82-8e93-e70d2bf62979 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:36.197895 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 291/589] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:36.198614 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6448d-a6b9-4b82-8e93-e70d2bf62979 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:36.229689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017e01b-755f-4c72-a860-f2b13ee1c29e None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:36.245994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:36.270217 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b7a66-a9ed-4f71-b3d5-493c5f841ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:36.270883 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b7a66-a9ed-4f71-b3d5-493c5f841ba5 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:36.271164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b7a66-a9ed-4f71-b3d5-493c5f841ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:36.271477 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b7a66-a9ed-4f71-b3d5-493c5f841ba5 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:36.272197 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b7a66-a9ed-4f71-b3d5-493c5f841ba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:36.272500 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b7a66-a9ed-4f71-b3d5-493c5f841ba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:36.274549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b7a66-a9ed-4f71-b3d5-493c5f841ba5 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:36.291533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 299/590] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:36.292513 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b7a66-a9ed-4f71-b3d5-493c5f841ba5 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:36.329787 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6448d-a6b9-4b82-8e93-e70d2bf62979 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:36.352067 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:36.368866 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e28d2d-f118-46fe-84c0-18ed414d2e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:36.370009 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e28d2d-f118-46fe-84c0-18ed414d2e7a None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:36.370511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e28d2d-f118-46fe-84c0-18ed414d2e7a None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:36.371073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e28d2d-f118-46fe-84c0-18ed414d2e7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:36.379312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e28d2d-f118-46fe-84c0-18ed414d2e7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:36.381114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e28d2d-f118-46fe-84c0-18ed414d2e7a None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:36.396791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-f1e28d2d-f118-46fe-84c0-18ed414d2e7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:36.396791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:36.396791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:36.396791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:36.396791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:36.396791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:36.396791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:36.396791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:36.396791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:36.396791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:36.396791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:36.396791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:36.396791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:36.396791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:36.396791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:36.397712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:36.397712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:36.397712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:36.397712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:36.397712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:36.397712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:36.397712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:36.397712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:36.397712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:36.397712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:36.397712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:36.397712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:36.397712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:36.397712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:36.397712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:36.397712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:36.397712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:36.398604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:36.398604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:36.398604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:36.398604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:36.398604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:36.398604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:36.398604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:36.398604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:36.405326 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 292/591] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:36.406391 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e28d2d-f118-46fe-84c0-18ed414d2e7a None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:36.422583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b7a66-a9ed-4f71-b3d5-493c5f841ba5 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:36.471573 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d51bc-2810-4fd2-a817-a289100d9989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:36.472845 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d51bc-2810-4fd2-a817-a289100d9989 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:36.473134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d51bc-2810-4fd2-a817-a289100d9989 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:36.473409 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d51bc-2810-4fd2-a817-a289100d9989 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:36.483384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d51bc-2810-4fd2-a817-a289100d9989 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:36.484838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d51bc-2810-4fd2-a817-a289100d9989 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:36.508994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-c19d51bc-2810-4fd2-a817-a289100d9989 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 22 11:32:36.508994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:36.508994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:36.508994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:36.508994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:36.508994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:36.508994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:36.508994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:36.508994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:36.508994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:36.508994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:36.508994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:36.508994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:36.508994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:36.508994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:36.508994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:36.509992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:36.509992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:36.509992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:36.509992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:36.509992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:36.509992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:36.509992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:36.509992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:36.509992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:36.509992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:36.509992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:36.509992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:36.509992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:36.509992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:36.509992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:36.509992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:36.509992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:36.509992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:36.510982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:36.510982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:36.510982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:36.510982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:36.510982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:36.510982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:36.510982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:36.510982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:36.510982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 22 11:32:36.510982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:36.510982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 300/592] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:36.510982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d51bc-2810-4fd2-a817-a289100d9989 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:36.519649 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e28d2d-f118-46fe-84c0-18ed414d2e7a None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:36.557813 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:36.560750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198e09f-a010-4cec-82b5-11cd7b286bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:36.561943 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198e09f-a010-4cec-82b5-11cd7b286bac None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:36.562465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198e09f-a010-4cec-82b5-11cd7b286bac None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:36.562964 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198e09f-a010-4cec-82b5-11cd7b286bac None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:36.578693 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d198e09f-a010-4cec-82b5-11cd7b286bac None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:36.578693 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d198e09f-a010-4cec-82b5-11cd7b286bac None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:36.593134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-d198e09f-a010-4cec-82b5-11cd7b286bac None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 22 11:32:36.593134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:36.593134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:36.593134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:36.593134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:36.593134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:36.593134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:36.593134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:36.593134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:36.593134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:36.593134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:36.593134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:36.593134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:36.593134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:36.593134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:36.593134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:36.594122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:36.594122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:36.594122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:36.594122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:36.594122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:36.594122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:36.594122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:36.594122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:36.594122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:36.594122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:36.594122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:36.594122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:36.594122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:36.594122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:36.594122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:36.594122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:36.594122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:36.594122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:36.595072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:36.595072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:36.595072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:36.595072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:36.595072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:36.595072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:36.595072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:36.595072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:36.595072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 22 11:32:36.595072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:36.595072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 293/593] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:36.599835 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198e09f-a010-4cec-82b5-11cd7b286bac None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:36.630739 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d51bc-2810-4fd2-a817-a289100d9989 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:36.645742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:36.667959 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f34891-8c09-4d15-97fe-829ab0e110b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:36.668428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f34891-8c09-4d15-97fe-829ab0e110b8 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:36.668718 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f34891-8c09-4d15-97fe-829ab0e110b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:36.668998 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f34891-8c09-4d15-97fe-829ab0e110b8 None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:36.669797 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f34891-8c09-4d15-97fe-829ab0e110b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:36.670110 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f34891-8c09-4d15-97fe-829ab0e110b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:36.671580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f34891-8c09-4d15-97fe-829ab0e110b8 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:36.684357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 301/594] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:36.685279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f34891-8c09-4d15-97fe-829ab0e110b8 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:36.710031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198e09f-a010-4cec-82b5-11cd7b286bac None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:36.745843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7194f7-5508-4329-9c5c-302d4b0180ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:36.746800 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7194f7-5508-4329-9c5c-302d4b0180ba None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:36.747443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7194f7-5508-4329-9c5c-302d4b0180ba None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:36.747860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7194f7-5508-4329-9c5c-302d4b0180ba None admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:36.748992 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7194f7-5508-4329-9c5c-302d4b0180ba None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:36.749454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7194f7-5508-4329-9c5c-302d4b0180ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:36.751071 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7194f7-5508-4329-9c5c-302d4b0180ba None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:36.772965 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 294/595] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Jun 22 11:32:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:36.774032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7194f7-5508-4329-9c5c-302d4b0180ba None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:36.790086 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:36.801015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f34891-8c09-4d15-97fe-829ab0e110b8 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:36.837150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60070c23-f373-4f97-acc2-d55b9150a2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:36.837857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60070c23-f373-4f97-acc2-d55b9150a2ff None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:36.838123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60070c23-f373-4f97-acc2-d55b9150a2ff None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:36.838465 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60070c23-f373-4f97-acc2-d55b9150a2ff None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:36.839258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60070c23-f373-4f97-acc2-d55b9150a2ff None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:36.839565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60070c23-f373-4f97-acc2-d55b9150a2ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:36.841005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60070c23-f373-4f97-acc2-d55b9150a2ff None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:36.865186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 302/596] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jun 22 11:32:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:36.865186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60070c23-f373-4f97-acc2-d55b9150a2ff None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:36.883239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:36.887879 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7194f7-5508-4329-9c5c-302d4b0180ba None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:36.935912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef14a7-9a1a-4f9a-9e6c-9aac1fa38228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:36.937057 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef14a7-9a1a-4f9a-9e6c-9aac1fa38228 None admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:36.937494 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef14a7-9a1a-4f9a-9e6c-9aac1fa38228 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:36.938054 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef14a7-9a1a-4f9a-9e6c-9aac1fa38228 None admin] PATH_INFO: `/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/7ed133721ed044708ee362c93f564d07/roles/64c2d69f15ca458c8267c1acfd42d9fc` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:36.958690 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef14a7-9a1a-4f9a-9e6c-9aac1fa38228 None admin] RBAC: Authorizing `identity:create_grant(project_id=47d2bb094c9843c9a7726f1d672d2433, user_id=7ed133721ed044708ee362c93f564d07, role_id=64c2d69f15ca458c8267c1acfd42d9fc)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:36.960818 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef14a7-9a1a-4f9a-9e6c-9aac1fa38228 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:36.976417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 295/597] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:32:36 2022] PUT /identity/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/7ed133721ed044708ee362c93f564d07/roles/64c2d69f15ca458c8267c1acfd42d9fc => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:32:36.977641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef14a7-9a1a-4f9a-9e6c-9aac1fa38228 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:36.982835 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60070c23-f373-4f97-acc2-d55b9150a2ff None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:37.028265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9fa83-b70c-4100-ac6d-00e950b40b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:37.029073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9fa83-b70c-4100-ac6d-00e950b40b95 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:37.029362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9fa83-b70c-4100-ac6d-00e950b40b95 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:37.029640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9fa83-b70c-4100-ac6d-00e950b40b95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:37.060629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9fa83-b70c-4100-ac6d-00e950b40b95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:37.062138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9fa83-b70c-4100-ac6d-00e950b40b95 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:37.099015 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef14a7-9a1a-4f9a-9e6c-9aac1fa38228 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:37.108026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-14c9fa83-b70c-4100-ac6d-00e950b40b95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:37.108026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:37.108026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:37.108026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:37.108026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:37.108026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:37.108026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:37.108026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:37.108026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:37.108026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:37.108026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:37.108026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:37.108026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:37.108026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:37.108026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:37.109174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:37.109174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:37.109174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:37.109174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:37.109174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:37.109174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:37.109174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:37.109174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:37.109174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:37.109174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:37.109174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:37.109174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:37.109174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:37.109174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:37.109174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:37.109174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:37.109174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:37.110262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:37.110262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:37.110262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:37.110262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:37.110262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 303/598] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:37.115532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9fa83-b70c-4100-ac6d-00e950b40b95 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:37.122099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:37.143933 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d8344-cc9b-4b6e-a913-05ea8c4a654b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:37.145785 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d8344-cc9b-4b6e-a913-05ea8c4a654b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:37.146416 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d8344-cc9b-4b6e-a913-05ea8c4a654b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:37.146824 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d8344-cc9b-4b6e-a913-05ea8c4a654b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:37.159203 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d8344-cc9b-4b6e-a913-05ea8c4a654b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:37.161339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d8344-cc9b-4b6e-a913-05ea8c4a654b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:37.171547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:37.186670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-710d8344-cc9b-4b6e-a913-05ea8c4a654b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:37.186670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:37.186670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:37.186670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:37.186670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:37.186670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:37.186670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:37.186670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:37.186670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:37.186670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:37.186670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:37.186670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:37.186670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:37.186670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:37.186670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:37.187799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:37.187799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:37.187799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:37.187799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:37.187799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:37.187799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:37.187799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:37.187799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:37.187799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:37.187799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:37.187799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:37.187799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:37.187799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:37.187799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:37.187799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:37.187799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:37.187799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:37.188765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:37.188765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:37.188765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:37.188765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:37.190529 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 296/599] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:37.196399 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d8344-cc9b-4b6e-a913-05ea8c4a654b None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:37.333377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9fa83-b70c-4100-ac6d-00e950b40b95 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:37.373334 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d8344-cc9b-4b6e-a913-05ea8c4a654b None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:37.381794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9400b-0e29-4c89-bf30-b63b6cfb4252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:37.383297 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9400b-0e29-4c89-bf30-b63b6cfb4252 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:37.383766 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9400b-0e29-4c89-bf30-b63b6cfb4252 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:37.384200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9400b-0e29-4c89-bf30-b63b6cfb4252 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:37.385528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9400b-0e29-4c89-bf30-b63b6cfb4252 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:37.385996 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9400b-0e29-4c89-bf30-b63b6cfb4252 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:37.387974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9400b-0e29-4c89-bf30-b63b6cfb4252 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:37.404639 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 304/600] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:37.406750 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd393059-3999-4192-9f67-fcb405d39178 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:37.407182 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd393059-3999-4192-9f67-fcb405d39178 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:37.408015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd393059-3999-4192-9f67-fcb405d39178 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:37.409984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 305/601] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:37.411042 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd393059-3999-4192-9f67-fcb405d39178 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:37.423606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30399554-bdfd-4bc2-9a25-9b9d417efd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:37.424582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30399554-bdfd-4bc2-9a25-9b9d417efd41 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:37.425006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30399554-bdfd-4bc2-9a25-9b9d417efd41 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:37.425582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30399554-bdfd-4bc2-9a25-9b9d417efd41 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:37.426451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30399554-bdfd-4bc2-9a25-9b9d417efd41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:37.426906 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30399554-bdfd-4bc2-9a25-9b9d417efd41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:37.428566 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30399554-bdfd-4bc2-9a25-9b9d417efd41 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:37.443979 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 297/602] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:37.445525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb13ca3-8c1b-4dfe-bffb-ce2efefd3aa4 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:37.445965 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb13ca3-8c1b-4dfe-bffb-ce2efefd3aa4 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:37.446731 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb13ca3-8c1b-4dfe-bffb-ce2efefd3aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:37.464997 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-abb13ca3-8c1b-4dfe-bffb-ce2efefd3aa4 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:37.466543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-abb13ca3-8c1b-4dfe-bffb-ce2efefd3aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:37.481930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:37.525835 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd393059-3999-4192-9f67-fcb405d39178 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:37.533010 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:37.545910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 298/603] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:37.546996 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb13ca3-8c1b-4dfe-bffb-ce2efefd3aa4 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:37.584938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c3e13-d475-4980-a117-39bccfc5bc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:37.585943 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c3e13-d475-4980-a117-39bccfc5bc1f None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:37.586324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c3e13-d475-4980-a117-39bccfc5bc1f None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:37.586695 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c3e13-d475-4980-a117-39bccfc5bc1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:37.595592 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c3e13-d475-4980-a117-39bccfc5bc1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:37.597777 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c3e13-d475-4980-a117-39bccfc5bc1f None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:37.614413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-4a5c3e13-d475-4980-a117-39bccfc5bc1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:37.614413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:37.614413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:37.614413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:37.614413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:37.614413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:37.614413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:37.614413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:37.614413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:37.614413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:37.614413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:37.614413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:37.614413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:37.614413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:37.614413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:37.615563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:37.615563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:37.615563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:37.615563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:37.615563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:37.615563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:37.615563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:37.615563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:37.615563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:37.615563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:37.615563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:37.615563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:37.615563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:37.615563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:37.615563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:37.615563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:37.615563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:37.616642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:37.616642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:37.616642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:37.616642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:37.616642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:37.616642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:37.616642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:37.616642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:37.618446 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 306/604] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:37.619339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c3e13-d475-4980-a117-39bccfc5bc1f None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:37.662809 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb13ca3-8c1b-4dfe-bffb-ce2efefd3aa4 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:37.700021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5410ff41-651e-4b3c-9e3b-651405e08135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:37.701052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410ff41-651e-4b3c-9e3b-651405e08135 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:37.701477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410ff41-651e-4b3c-9e3b-651405e08135 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:37.701908 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410ff41-651e-4b3c-9e3b-651405e08135 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:37.709904 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5410ff41-651e-4b3c-9e3b-651405e08135 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:37.711698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5410ff41-651e-4b3c-9e3b-651405e08135 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:37.725772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-5410ff41-651e-4b3c-9e3b-651405e08135 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:37.725772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:37.725772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:37.725772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:37.725772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:37.725772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:37.725772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:37.725772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:37.725772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:37.725772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:37.725772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:37.725772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:37.725772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:37.725772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:37.725772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:37.726889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:37.726889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:37.726889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:37.726889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:37.726889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:37.726889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:37.726889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:37.726889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:37.726889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:37.726889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:37.726889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:37.726889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:37.726889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:37.726889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:37.726889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:37.726889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:37.726889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:37.727986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:37.727986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:37.727986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:37.727986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:37.727986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:37.727986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:37.727986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:37.727986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:37.730082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 299/605] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:37.731007 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5410ff41-651e-4b3c-9e3b-651405e08135 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:37.733596 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c3e13-d475-4980-a117-39bccfc5bc1f None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:37.769103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:37.772978 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3fe35-a300-465a-b0d9-d61d3bed38af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:37.774422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3fe35-a300-465a-b0d9-d61d3bed38af None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:37.774594 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3fe35-a300-465a-b0d9-d61d3bed38af None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:37.774932 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3fe35-a300-465a-b0d9-d61d3bed38af None admin] PATH_INFO: `/v3/roles/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:37.785889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:37.788871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3fe35-a300-465a-b0d9-d61d3bed38af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:37.798382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-49a3fe35-a300-465a-b0d9-d61d3bed38af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:37.798382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:37.798382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:37.798382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:37.798382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:37.798382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:37.798382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:37.798382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:37.798382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:37.798382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:37.798382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:37.798382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:37.798382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:37.798382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:37.798382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:37.798382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:37.799538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:37.799538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:37.799538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:37.799538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:37.799538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:37.799538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:37.799538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:37.799538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:37.799538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:37.799538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:37.799538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:37.799538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:37.799538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:37.799538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:37.799538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:37.799538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:37.801106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:37.801106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:37.801106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:37.801106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:37.801106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:37.801106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:37.801106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:37.801106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:37.801106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:37.801106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:37.801106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:37.801106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:37.807684 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 307/606] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:37.808766 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3fe35-a300-465a-b0d9-d61d3bed38af None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:37.843429 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5410ff41-651e-4b3c-9e3b-651405e08135 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:37.880400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa0a5b-ed4f-408b-a43b-3633c107ca33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:37.881566 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa0a5b-ed4f-408b-a43b-3633c107ca33 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:37.881995 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa0a5b-ed4f-408b-a43b-3633c107ca33 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:37.882415 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa0a5b-ed4f-408b-a43b-3633c107ca33 None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:37.883389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa0a5b-ed4f-408b-a43b-3633c107ca33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:37.883897 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa0a5b-ed4f-408b-a43b-3633c107ca33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:37.885770 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa0a5b-ed4f-408b-a43b-3633c107ca33 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:37.899965 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 300/607] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:37.901229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa0a5b-ed4f-408b-a43b-3633c107ca33 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:37.921133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3fe35-a300-465a-b0d9-d61d3bed38af None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:37.971833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b437d43-3b06-4a74-84c2-f943d83e545e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:37.972464 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b437d43-3b06-4a74-84c2-f943d83e545e None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:37.972753 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b437d43-3b06-4a74-84c2-f943d83e545e None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:37.973035 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b437d43-3b06-4a74-84c2-f943d83e545e None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:37.974154 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b437d43-3b06-4a74-84c2-f943d83e545e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:37.974469 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b437d43-3b06-4a74-84c2-f943d83e545e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:37.976271 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b437d43-3b06-4a74-84c2-f943d83e545e None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:37.994847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 308/608] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:37.995867 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b437d43-3b06-4a74-84c2-f943d83e545e None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:38.024613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa0a5b-ed4f-408b-a43b-3633c107ca33 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:38.036533 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:38.057212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:38.064099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d14f0-24ea-49e9-9ec1-12b8bd77bc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:38.064965 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d14f0-24ea-49e9-9ec1-12b8bd77bc2f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:38.065354 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d14f0-24ea-49e9-9ec1-12b8bd77bc2f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:38.065737 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d14f0-24ea-49e9-9ec1-12b8bd77bc2f None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:38.067175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d14f0-24ea-49e9-9ec1-12b8bd77bc2f None admin] RBAC: Adding query filter params (name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:38.067715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d14f0-24ea-49e9-9ec1-12b8bd77bc2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:38.069335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d14f0-24ea-49e9-9ec1-12b8bd77bc2f None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:38.078953 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 301/609] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:32:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:38.079856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d14f0-24ea-49e9-9ec1-12b8bd77bc2f None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:38.143668 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b437d43-3b06-4a74-84c2-f943d83e545e None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:38.190080 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d14f0-24ea-49e9-9ec1-12b8bd77bc2f None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:38.193763 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45b5b0-370c-45a0-8d26-3e6ef372fb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:38.194872 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45b5b0-370c-45a0-8d26-3e6ef372fb52 None admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:38.195302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45b5b0-370c-45a0-8d26-3e6ef372fb52 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:38.195747 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45b5b0-370c-45a0-8d26-3e6ef372fb52 None admin] PATH_INFO: `/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/c00d0bb777a044a3a4d4540d187d938a/roles/64c2d69f15ca458c8267c1acfd42d9fc` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:38.216853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45b5b0-370c-45a0-8d26-3e6ef372fb52 None admin] RBAC: Authorizing `identity:create_grant(project_id=47d2bb094c9843c9a7726f1d672d2433, user_id=c00d0bb777a044a3a4d4540d187d938a, role_id=64c2d69f15ca458c8267c1acfd42d9fc)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:38.219265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a45b5b0-370c-45a0-8d26-3e6ef372fb52 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:38.230911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21c62d-db27-4a4e-b407-a519909d0cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:38.231782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21c62d-db27-4a4e-b407-a519909d0cef None admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:38.232066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21c62d-db27-4a4e-b407-a519909d0cef None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:38.232346 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21c62d-db27-4a4e-b407-a519909d0cef None admin] PATH_INFO: `/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/ceb170ac6353495b9d1dc6723d59567a/roles/64c2d69f15ca458c8267c1acfd42d9fc` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:38.245574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 309/610] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:32:37 2022] PUT /identity/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/c00d0bb777a044a3a4d4540d187d938a/roles/64c2d69f15ca458c8267c1acfd42d9fc => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:32:38.245574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45b5b0-370c-45a0-8d26-3e6ef372fb52 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:38.261482 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21c62d-db27-4a4e-b407-a519909d0cef None admin] RBAC: Authorizing `identity:create_grant(project_id=47d2bb094c9843c9a7726f1d672d2433, user_id=ceb170ac6353495b9d1dc6723d59567a, role_id=64c2d69f15ca458c8267c1acfd42d9fc)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:38.262987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21c62d-db27-4a4e-b407-a519909d0cef None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:38.285213 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 302/611] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:32:38 2022] PUT /identity/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/ceb170ac6353495b9d1dc6723d59567a/roles/64c2d69f15ca458c8267c1acfd42d9fc => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:32:38.376855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d1fbe-3532-42d4-b54f-c0afa36c9423 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:38.377149 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d1fbe-3532-42d4-b54f-c0afa36c9423 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:38.377436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d1fbe-3532-42d4-b54f-c0afa36c9423 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:38.379468 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 303/612] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:38.398003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487eae8c-393a-4c61-8a09-e28158d8810c None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:38.398509 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487eae8c-393a-4c61-8a09-e28158d8810c None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:38.398902 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487eae8c-393a-4c61-8a09-e28158d8810c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:38.410950 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:38.426988 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45b5b0-370c-45a0-8d26-3e6ef372fb52 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:38.432544 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-487eae8c-393a-4c61-8a09-e28158d8810c None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:38.433974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-487eae8c-393a-4c61-8a09-e28158d8810c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:38.475509 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:38.505846 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff1f47-a2d9-4564-86d5-75e26ff44f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:38.506960 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff1f47-a2d9-4564-86d5-75e26ff44f16 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:38.507429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff1f47-a2d9-4564-86d5-75e26ff44f16 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:38.507856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff1f47-a2d9-4564-86d5-75e26ff44f16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:38.520446 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff1f47-a2d9-4564-86d5-75e26ff44f16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:38.522210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff1f47-a2d9-4564-86d5-75e26ff44f16 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:38.538585 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 304/613] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:38.542530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-6cff1f47-a2d9-4564-86d5-75e26ff44f16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:38.542530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:38.542530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:38.542530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:38.542530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:38.542530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:38.542530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:38.542530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:38.542530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:38.542530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:38.542530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:38.542530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:38.542530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:38.542530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:38.542530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:38.543487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:38.543487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:38.543487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:38.543487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:38.543487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:38.543487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:38.543487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:38.543487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:38.543487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:38.543487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:38.543487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:38.543487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:38.543487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:38.543487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:38.543487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:38.543487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:38.543487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:38.544362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:38.544362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:38.544362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:38.544362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:38.544362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 310/614] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:38.549117 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487eae8c-393a-4c61-8a09-e28158d8810c None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:38.578505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff1f47-a2d9-4564-86d5-75e26ff44f16 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:38.661396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487eae8c-393a-4c61-8a09-e28158d8810c None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:38.701989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a47731-2cae-4274-9d85-fc73a572ce62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:38.703099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a47731-2cae-4274-9d85-fc73a572ce62 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:38.704330 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a47731-2cae-4274-9d85-fc73a572ce62 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:38.704764 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a47731-2cae-4274-9d85-fc73a572ce62 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:38.705630 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a47731-2cae-4274-9d85-fc73a572ce62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:38.706072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a47731-2cae-4274-9d85-fc73a572ce62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:38.708182 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a47731-2cae-4274-9d85-fc73a572ce62 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:38.721052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 305/615] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:38.731483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a47731-2cae-4274-9d85-fc73a572ce62 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:38.757858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:38.768227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff1f47-a2d9-4564-86d5-75e26ff44f16 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:38.836971 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d9bb0-328a-484f-8810-ff3a7f37b89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:38.837779 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d9bb0-328a-484f-8810-ff3a7f37b89b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:38.838062 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d9bb0-328a-484f-8810-ff3a7f37b89b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:38.838337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d9bb0-328a-484f-8810-ff3a7f37b89b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:38.856099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d9bb0-328a-484f-8810-ff3a7f37b89b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:38.868222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-1f3d9bb0-328a-484f-8810-ff3a7f37b89b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:38.868222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:38.868222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:38.868222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:38.868222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:38.868222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:38.868222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:38.868222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:38.868222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:38.868222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:38.868222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:38.868222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:38.868222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:38.868222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:38.868222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:38.868222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:38.869398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:38.869398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:38.869398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:38.869398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:38.869398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:38.869398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:38.869398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:38.869398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:38.869398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:38.869398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:38.869398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:38.869398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:38.869398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:38.869398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:38.869398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:38.869398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:38.870450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:38.870450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:38.870450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:38.870450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:38.870450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:38.870450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:38.870450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:38.870450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:38.870450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:38.870450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:38.870450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:38.870450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:38.877188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 311/616] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:32:38.883675 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d9bb0-328a-484f-8810-ff3a7f37b89b None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:38.897083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:38.928543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a47731-2cae-4274-9d85-fc73a572ce62 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:38.977891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b6bff-2bb7-4f0a-a6ef-d717403a56cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:38.978703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b6bff-2bb7-4f0a-a6ef-d717403a56cd None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:38.978988 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b6bff-2bb7-4f0a-a6ef-d717403a56cd None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:38.979267 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b6bff-2bb7-4f0a-a6ef-d717403a56cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:38.988174 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b6bff-2bb7-4f0a-a6ef-d717403a56cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:38.989736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b6bff-2bb7-4f0a-a6ef-d717403a56cd None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:39.005406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-c13b6bff-2bb7-4f0a-a6ef-d717403a56cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 22 11:32:39.005406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:39.005406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:39.005406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:39.005406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:39.005406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:39.005406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:39.005406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:39.005406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:39.005406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:39.005406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:39.005406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:39.005406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:39.005406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:39.005406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:39.005406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:39.006443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:39.006443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:39.006443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:39.006443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:39.006443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:39.006443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:39.006443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:39.006443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:39.006443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:39.006443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:39.006443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:39.006443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:39.006443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:39.006443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:39.006443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:39.006443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:39.006443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:39.006443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:39.007449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:39.007449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:39.007449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:39.007449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:39.007449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:39.007449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:39.007449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:39.007449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:39.007449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 22 11:32:39.007449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:39.007449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 306/617] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jun 22 11:32:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:39.012647 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b6bff-2bb7-4f0a-a6ef-d717403a56cd None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:39.036323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d9bb0-328a-484f-8810-ff3a7f37b89b None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:39.124668 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:39.126342 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717a389-af41-4bd0-8d6a-704d72430600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:39.126814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717a389-af41-4bd0-8d6a-704d72430600 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:39.127207 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717a389-af41-4bd0-8d6a-704d72430600 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:39.127591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717a389-af41-4bd0-8d6a-704d72430600 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:39.128905 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717a389-af41-4bd0-8d6a-704d72430600 None admin] RBAC: Adding query filter params (name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:39.129220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717a389-af41-4bd0-8d6a-704d72430600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:39.130611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717a389-af41-4bd0-8d6a-704d72430600 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:39.147601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 312/618] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:32:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:39.154812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717a389-af41-4bd0-8d6a-704d72430600 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:39.166346 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b6bff-2bb7-4f0a-a6ef-d717403a56cd None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:39.202812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4a6b7-5ae4-4660-b06e-d5bd38a1e421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:39.203916 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4a6b7-5ae4-4660-b06e-d5bd38a1e421 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:39.204566 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4a6b7-5ae4-4660-b06e-d5bd38a1e421 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:39.205063 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4a6b7-5ae4-4660-b06e-d5bd38a1e421 None admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:39.206197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4a6b7-5ae4-4660-b06e-d5bd38a1e421 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:39.207305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4a6b7-5ae4-4660-b06e-d5bd38a1e421 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:39.209216 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4a6b7-5ae4-4660-b06e-d5bd38a1e421 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:39.228819 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 307/619] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jun 22 11:32:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:39.234801 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4a6b7-5ae4-4660-b06e-d5bd38a1e421 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:39.341135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4a6b7-5ae4-4660-b06e-d5bd38a1e421 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:39.356808 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:39.377167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:39.384515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3c9d36-e8de-408d-ae79-4716faabf3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:39.385663 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c9d36-e8de-408d-ae79-4716faabf3a3 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:39.386168 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c9d36-e8de-408d-ae79-4716faabf3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:39.386708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3c9d36-e8de-408d-ae79-4716faabf3a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:39.403584 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717a389-af41-4bd0-8d6a-704d72430600 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:39.439550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953d005-7b03-4b9e-b72a-c430e1860773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:39.440338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953d005-7b03-4b9e-b72a-c430e1860773 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:39.440626 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953d005-7b03-4b9e-b72a-c430e1860773 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:39.440928 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953d005-7b03-4b9e-b72a-c430e1860773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:39.453006 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953d005-7b03-4b9e-b72a-c430e1860773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:39.454556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953d005-7b03-4b9e-b72a-c430e1860773 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:39.465017 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3c9d36-e8de-408d-ae79-4716faabf3a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:39.466807 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3c9d36-e8de-408d-ae79-4716faabf3a3 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:39.473856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-7953d005-7b03-4b9e-b72a-c430e1860773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:39.473856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:39.473856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:39.473856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:39.473856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:39.473856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:39.473856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:39.473856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:39.473856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:39.473856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:39.473856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:39.473856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:39.473856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:39.473856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:39.473856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:39.474781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:39.474781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:39.474781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:39.474781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:39.474781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:39.474781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:39.474781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:39.474781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:39.474781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:39.474781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:39.474781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:39.474781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:39.474781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:39.474781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:39.474781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:39.474781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:39.474781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:39.475689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:39.475689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:39.475689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:39.475689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:39.475689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 313/620] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:39.476387 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4feb42-3f50-4a47-af51-7231f872d0ee None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:39.476669 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4feb42-3f50-4a47-af51-7231f872d0ee None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:39.476941 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4feb42-3f50-4a47-af51-7231f872d0ee None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:39.482529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 314/621] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:39.483489 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4feb42-3f50-4a47-af51-7231f872d0ee None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:39.584528 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 308/622] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:39.586368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b86139-8078-4ef1-8a4a-c7145cc7a13d None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:39.586885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b86139-8078-4ef1-8a4a-c7145cc7a13d None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:39.587409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b86139-8078-4ef1-8a4a-c7145cc7a13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:39.595804 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4feb42-3f50-4a47-af51-7231f872d0ee None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:39.606666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-84b86139-8078-4ef1-8a4a-c7145cc7a13d None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:39.608446 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-84b86139-8078-4ef1-8a4a-c7145cc7a13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:39.615546 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:39.637354 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222de5d0-03af-4676-a092-035bcca67605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:39.638050 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222de5d0-03af-4676-a092-035bcca67605 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:39.638333 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222de5d0-03af-4676-a092-035bcca67605 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:39.638611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222de5d0-03af-4676-a092-035bcca67605 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:39.639403 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222de5d0-03af-4676-a092-035bcca67605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:39.639717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222de5d0-03af-4676-a092-035bcca67605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:39.641567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222de5d0-03af-4676-a092-035bcca67605 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:39.642510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:39.658175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 315/623] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:39.659424 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222de5d0-03af-4676-a092-035bcca67605 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:39.699371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 309/624] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:39.699371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b86139-8078-4ef1-8a4a-c7145cc7a13d None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:39.790469 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222de5d0-03af-4676-a092-035bcca67605 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:39.801099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b86139-8078-4ef1-8a4a-c7145cc7a13d None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:39.825888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8994c-beaa-4056-8b21-2b16b8553f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:39.826933 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8994c-beaa-4056-8b21-2b16b8553f9e None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:39.827385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8994c-beaa-4056-8b21-2b16b8553f9e None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:39.827829 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8994c-beaa-4056-8b21-2b16b8553f9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:39.838500 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07efdea2-5830-40b0-8019-42b34db3445f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:39.839213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8994c-beaa-4056-8b21-2b16b8553f9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:39.839548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efdea2-5830-40b0-8019-42b34db3445f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:39.839934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efdea2-5830-40b0-8019-42b34db3445f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:39.840327 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efdea2-5830-40b0-8019-42b34db3445f None admin] PATH_INFO: `/v3/users/swift` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:39.841322 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8994c-beaa-4056-8b21-2b16b8553f9e None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:39.848756 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07efdea2-5830-40b0-8019-42b34db3445f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:39.850379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07efdea2-5830-40b0-8019-42b34db3445f None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:39.859405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-77c8994c-beaa-4056-8b21-2b16b8553f9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:39.859405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:39.859405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:39.859405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:39.859405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:39.859405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:39.859405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:39.859405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:39.859405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:39.859405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:39.859405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:39.859405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:39.859405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:39.859405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:39.859405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:39.860435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:39.860435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:39.860435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:39.860435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:39.860435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:39.860435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:39.860435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:39.860435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:39.860435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:39.860435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:39.860435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:39.860435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:39.860435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:39.860435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:39.860435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:39.860435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:39.860435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:39.861449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:39.861449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:39.861449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:39.861449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:39.863367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 316/625] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:39.864281 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8994c-beaa-4056-8b21-2b16b8553f9e None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:39.878931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-07efdea2-5830-40b0-8019-42b34db3445f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 22 11:32:39.878931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:39.878931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:39.878931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:39.878931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:39.878931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:39.878931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:39.878931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:39.878931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:39.878931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:39.878931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:39.878931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:39.878931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:39.878931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:39.878931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:39.878931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:39.880127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:39.880127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:39.880127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:39.880127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:39.880127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:39.880127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:39.880127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:39.880127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:39.880127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:39.880127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:39.880127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:39.880127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:39.880127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:39.880127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:39.880127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:39.880127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:39.880127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:39.880127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:39.881249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:39.881249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:39.881249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:39.881249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:39.881249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:39.881249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:39.881249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:39.881249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:39.881249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 22 11:32:39.881249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:39.882982 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 310/626] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:32:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:39.884059 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07efdea2-5830-40b0-8019-42b34db3445f None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:39.902881 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:39.919702 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:39.991665 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8994c-beaa-4056-8b21-2b16b8553f9e None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:40.014081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07efdea2-5830-40b0-8019-42b34db3445f None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:40.032754 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6f2fe-012e-482a-ab01-b310aa49f778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:40.033979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6f2fe-012e-482a-ab01-b310aa49f778 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:40.034368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6f2fe-012e-482a-ab01-b310aa49f778 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:40.034782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6f2fe-012e-482a-ab01-b310aa49f778 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:40.051702 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6f2fe-012e-482a-ab01-b310aa49f778 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:40.056038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ff778-ace3-48d2-8e44-f809a37b55e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:40.056908 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ff778-ace3-48d2-8e44-f809a37b55e0 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:40.057399 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ff778-ace3-48d2-8e44-f809a37b55e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:40.057873 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ff778-ace3-48d2-8e44-f809a37b55e0 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:40.058772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ff778-ace3-48d2-8e44-f809a37b55e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:40.059251 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ff778-ace3-48d2-8e44-f809a37b55e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:40.060591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-09f6f2fe-012e-482a-ab01-b310aa49f778 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:40.060591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:40.060591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:40.060591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:40.060591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:40.060591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:40.060591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:40.060591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:40.060591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:40.060591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:40.060591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:40.060591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:40.060591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:40.060591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:40.060591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:40.060591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:40.061488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:40.061488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:40.061488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:40.061488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:40.061488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:40.061488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:40.061488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:40.061488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:40.061488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:40.061488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:40.061488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:40.061488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:40.061488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:40.061488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:40.061488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:40.061488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:40.062304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:40.062304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:40.062304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:40.062304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:40.062304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:40.062304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:40.062304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:40.062304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:40.062304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:40.062304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:40.062304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:40.062304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:40.065058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ff778-ace3-48d2-8e44-f809a37b55e0 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:40.066813 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 317/627] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:40.068047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6f2fe-012e-482a-ab01-b310aa49f778 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:40.079733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 311/628] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:40.080984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ff778-ace3-48d2-8e44-f809a37b55e0 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:40.166804 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:40.180866 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:40.191496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ff778-ace3-48d2-8e44-f809a37b55e0 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:40.192693 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6f2fe-012e-482a-ab01-b310aa49f778 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:40.228731 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f395ad-800c-4e76-b1a4-8f6f957f6bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:40.229532 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f395ad-800c-4e76-b1a4-8f6f957f6bf0 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:40.229925 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f395ad-800c-4e76-b1a4-8f6f957f6bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:40.230396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f395ad-800c-4e76-b1a4-8f6f957f6bf0 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:40.231020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd3316-4d24-4eb0-a7b8-dde9aa7eefc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:40.231329 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f395ad-800c-4e76-b1a4-8f6f957f6bf0 None admin] RBAC: Adding query filter params (name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:40.231788 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f395ad-800c-4e76-b1a4-8f6f957f6bf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:40.232310 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd3316-4d24-4eb0-a7b8-dde9aa7eefc5 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:40.232781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd3316-4d24-4eb0-a7b8-dde9aa7eefc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:40.233233 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd3316-4d24-4eb0-a7b8-dde9aa7eefc5 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:40.233468 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f395ad-800c-4e76-b1a4-8f6f957f6bf0 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:40.234430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd3316-4d24-4eb0-a7b8-dde9aa7eefc5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:40.235035 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd3316-4d24-4eb0-a7b8-dde9aa7eefc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:40.236792 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd3316-4d24-4eb0-a7b8-dde9aa7eefc5 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:40.247948 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 312/629] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:32:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:40.248711 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f395ad-800c-4e76-b1a4-8f6f957f6bf0 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:40.256663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 318/630] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jun 22 11:32:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:40.257880 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd3316-4d24-4eb0-a7b8-dde9aa7eefc5 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:40.360645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f395ad-800c-4e76-b1a4-8f6f957f6bf0 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:40.370003 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd3316-4d24-4eb0-a7b8-dde9aa7eefc5 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:40.398408 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c47d38-1686-4409-b2c2-a7af4ab7e7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:40.399308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c47d38-1686-4409-b2c2-a7af4ab7e7f7 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:40.399702 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c47d38-1686-4409-b2c2-a7af4ab7e7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:40.400174 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c47d38-1686-4409-b2c2-a7af4ab7e7f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:40.407718 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c47d38-1686-4409-b2c2-a7af4ab7e7f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:40.409374 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c47d38-1686-4409-b2c2-a7af4ab7e7f7 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:40.410220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6a16c-0c27-4663-ac25-6905bb53340b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:40.411396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6a16c-0c27-4663-ac25-6905bb53340b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:40.411969 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6a16c-0c27-4663-ac25-6905bb53340b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:40.412421 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6a16c-0c27-4663-ac25-6905bb53340b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:40.423211 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6a16c-0c27-4663-ac25-6905bb53340b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:40.424898 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6a16c-0c27-4663-ac25-6905bb53340b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:40.425844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-b4c47d38-1686-4409-b2c2-a7af4ab7e7f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:40.425844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:40.425844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:40.425844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:40.425844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:40.425844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:40.425844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:40.425844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:40.425844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:40.425844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:40.425844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:40.425844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:40.425844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:40.425844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:40.425844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:40.426696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:40.426696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:40.426696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:40.426696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:40.426696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:40.426696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:40.426696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:40.426696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:40.426696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:40.426696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:40.426696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:40.426696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:40.426696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:40.426696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:40.426696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:40.426696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:40.426696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:40.427633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:40.427633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:40.427633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:40.427633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:40.427633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:40.427633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:40.427633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:40.427633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:40.431806 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 313/631] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:40.432661 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c47d38-1686-4409-b2c2-a7af4ab7e7f7 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:40.435593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:40.438435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:40.450475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-5df6a16c-0c27-4663-ac25-6905bb53340b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:40.450475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:40.450475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:40.450475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:40.450475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:40.450475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:40.450475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:40.450475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:40.450475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:40.450475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:40.450475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:40.450475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:40.450475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:40.450475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:40.450475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:40.452046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:40.452046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:40.452046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:40.452046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:40.452046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:40.452046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:40.452046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:40.452046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:40.452046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:40.452046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:40.452046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:40.452046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:40.452046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:40.452046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:40.452046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:40.452046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:40.452046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:40.453092 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:40.453092 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:40.453092 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:40.453092 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:40.459462 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 319/632] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:40.459462 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6a16c-0c27-4663-ac25-6905bb53340b None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:40.556160 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c47d38-1686-4409-b2c2-a7af4ab7e7f7 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:40.564893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6a16c-0c27-4663-ac25-6905bb53340b None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:40.593832 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfca518-f96e-4680-9b60-1213ea0cf061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:40.594623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfca518-f96e-4680-9b60-1213ea0cf061 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:40.594912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfca518-f96e-4680-9b60-1213ea0cf061 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:40.595184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfca518-f96e-4680-9b60-1213ea0cf061 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:40.606638 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfca518-f96e-4680-9b60-1213ea0cf061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:40.607164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6c8fa-76d8-4e32-b50d-9b7efa55c719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:40.607871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6c8fa-76d8-4e32-b50d-9b7efa55c719 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:40.608157 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6c8fa-76d8-4e32-b50d-9b7efa55c719 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:40.608247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfca518-f96e-4680-9b60-1213ea0cf061 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:40.608435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6c8fa-76d8-4e32-b50d-9b7efa55c719 None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:40.609327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6c8fa-76d8-4e32-b50d-9b7efa55c719 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:40.609635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6c8fa-76d8-4e32-b50d-9b7efa55c719 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:40.611043 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6c8fa-76d8-4e32-b50d-9b7efa55c719 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:40.623546 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 320/633] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:40.628472 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6c8fa-76d8-4e32-b50d-9b7efa55c719 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:40.628648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-edfca518-f96e-4680-9b60-1213ea0cf061 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:40.628648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:40.628648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:40.628648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:40.628648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:40.628648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:40.628648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:40.628648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:40.628648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:40.628648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:40.628648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:40.628648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:40.628648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:40.628648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:40.628648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:40.629705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:40.629705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:40.629705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:40.629705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:40.629705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:40.629705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:40.629705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:40.629705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:40.629705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:40.629705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:40.629705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:40.629705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:40.629705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:40.629705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:40.629705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:40.629705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:40.629705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:40.630754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:40.630754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:40.630754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:40.630754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:40.630754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 314/634] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:40.630754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f8b99-73f5-4ec8-9738-566fed602003 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:40.630754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f8b99-73f5-4ec8-9738-566fed602003 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:40.630754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f8b99-73f5-4ec8-9738-566fed602003 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:40.631767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 315/635] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:40.633191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e16b1-4e53-4c09-97ac-f1e4610e0475 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:40.633477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e16b1-4e53-4c09-97ac-f1e4610e0475 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:40.633753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e16b1-4e53-4c09-97ac-f1e4610e0475 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:40.635138 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 316/636] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:40.635912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e16b1-4e53-4c09-97ac-f1e4610e0475 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:40.687181 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:40.692846 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:40.730837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6c8fa-76d8-4e32-b50d-9b7efa55c719 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:40.739516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e16b1-4e53-4c09-97ac-f1e4610e0475 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:40.763249 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d86b5-0d18-4677-8d6d-2d06ace550b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:40.763940 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d86b5-0d18-4677-8d6d-2d06ace550b4 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:40.764226 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d86b5-0d18-4677-8d6d-2d06ace550b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:40.764615 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d86b5-0d18-4677-8d6d-2d06ace550b4 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:40.765493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d86b5-0d18-4677-8d6d-2d06ace550b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:40.765803 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d86b5-0d18-4677-8d6d-2d06ace550b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:40.767181 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d86b5-0d18-4677-8d6d-2d06ace550b4 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:40.773888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41f04c-d27e-4038-b1bd-d038f38252c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:40.774545 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41f04c-d27e-4038-b1bd-d038f38252c6 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:40.774832 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41f04c-d27e-4038-b1bd-d038f38252c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:40.775111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41f04c-d27e-4038-b1bd-d038f38252c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:40.775975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41f04c-d27e-4038-b1bd-d038f38252c6 None admin] RBAC: Adding query filter params (role.id=64c2d69f15ca458c8267c1acfd42d9fc, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=479a376dcac14c509c1c55f0046d88cf) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:40.776291 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41f04c-d27e-4038-b1bd-d038f38252c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:40.777812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41f04c-d27e-4038-b1bd-d038f38252c6 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:40.779761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 321/637] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:40.780404 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d86b5-0d18-4677-8d6d-2d06ace550b4 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:40.801279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 317/638] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:32:40 2022] GET /identity/v3/role_assignments?user.id=479a376dcac14c509c1c55f0046d88cf&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=64c2d69f15ca458c8267c1acfd42d9fc => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:40.801279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00233eaf-1543-4d56-bebc-6bb1a473094b None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:40.801279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00233eaf-1543-4d56-bebc-6bb1a473094b None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:40.801279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00233eaf-1543-4d56-bebc-6bb1a473094b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:40.816982 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-00233eaf-1543-4d56-bebc-6bb1a473094b None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:40.818250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-00233eaf-1543-4d56-bebc-6bb1a473094b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:40.895785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d86b5-0d18-4677-8d6d-2d06ace550b4 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:40.899019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 318/639] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:40.900489 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73d6a0-9a55-4db8-9d0b-b191a689a370 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:40.900782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73d6a0-9a55-4db8-9d0b-b191a689a370 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:40.901065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73d6a0-9a55-4db8-9d0b-b191a689a370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:40.925343 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:40.926765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-2f73d6a0-9a55-4db8-9d0b-b191a689a370 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:40.928275 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-2f73d6a0-9a55-4db8-9d0b-b191a689a370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:40.956923 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064eddf-9f73-4c89-90b6-69ce8612a2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:40.958570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064eddf-9f73-4c89-90b6-69ce8612a2ed None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:40.959032 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064eddf-9f73-4c89-90b6-69ce8612a2ed None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:40.959249 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064eddf-9f73-4c89-90b6-69ce8612a2ed None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:40.960116 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064eddf-9f73-4c89-90b6-69ce8612a2ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:40.960427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064eddf-9f73-4c89-90b6-69ce8612a2ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:40.961838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064eddf-9f73-4c89-90b6-69ce8612a2ed None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:40.980577 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 322/640] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:40.981515 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064eddf-9f73-4c89-90b6-69ce8612a2ed None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:40.986656 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:41.018363 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 319/641] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:41.020091 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73d6a0-9a55-4db8-9d0b-b191a689a370 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:41.096569 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064eddf-9f73-4c89-90b6-69ce8612a2ed None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:41.127718 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73d6a0-9a55-4db8-9d0b-b191a689a370 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:41.133369 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da981caf-7844-4ae0-abd2-cbcb3e871f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:41.134168 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da981caf-7844-4ae0-abd2-cbcb3e871f06 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:41.134446 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da981caf-7844-4ae0-abd2-cbcb3e871f06 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:41.134717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da981caf-7844-4ae0-abd2-cbcb3e871f06 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:41.144312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da981caf-7844-4ae0-abd2-cbcb3e871f06 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:41.145739 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da981caf-7844-4ae0-abd2-cbcb3e871f06 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:41.160379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-da981caf-7844-4ae0-abd2-cbcb3e871f06 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 22 11:32:41.160379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:41.160379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:41.160379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:41.160379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:41.160379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:41.160379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:41.160379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:41.160379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:41.160379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:41.160379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:41.160379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:41.160379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:41.160379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:41.160379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:41.160379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:41.161650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:41.161650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:41.161650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:41.161650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:41.161650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:41.161650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:41.161650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:41.161650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:41.161650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:41.161650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:41.161650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:41.161650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:41.161650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:41.161650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:41.161650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:41.161650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:41.161650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:41.161650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:41.163166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:41.163166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:41.163166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:41.163166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:41.163166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:41.163166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:41.163166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:41.163166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:41.163166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 22 11:32:41.163166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:41.163166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 323/642] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:41.163166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da981caf-7844-4ae0-abd2-cbcb3e871f06 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:41.170681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ef6a2-6c30-48a4-8330-f077a23c08b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:41.171562 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ef6a2-6c30-48a4-8330-f077a23c08b7 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:41.171853 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ef6a2-6c30-48a4-8330-f077a23c08b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:41.172132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ef6a2-6c30-48a4-8330-f077a23c08b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:41.184114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ef6a2-6c30-48a4-8330-f077a23c08b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:41.192464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-500ef6a2-6c30-48a4-8330-f077a23c08b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:41.192464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:41.192464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:41.192464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:41.192464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:41.192464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:41.192464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:41.192464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:41.192464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:41.192464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:41.192464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:41.192464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:41.192464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:41.192464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:41.192464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:41.192464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:41.193465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:41.193465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:41.193465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:41.193465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:41.193465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:41.193465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:41.193465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:41.193465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:41.193465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:41.193465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:41.193465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:41.193465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:41.193465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:41.193465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:41.193465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:41.193465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:41.194375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:41.194375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:41.194375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:41.194375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:41.194375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:41.194375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:41.194375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:41.194375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:41.194375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:41.194375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:41.194375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:41.194375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:41.194375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 320/643] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:41.195092 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ef6a2-6c30-48a4-8330-f077a23c08b7 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:41.215177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:41.241565 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:41.281619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da981caf-7844-4ae0-abd2-cbcb3e871f06 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:41.313858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ef6a2-6c30-48a4-8330-f077a23c08b7 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:41.319406 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a6d24-5824-48eb-9119-5f5d0f898aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:41.320313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a6d24-5824-48eb-9119-5f5d0f898aca None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:41.320496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a6d24-5824-48eb-9119-5f5d0f898aca None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:41.320767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a6d24-5824-48eb-9119-5f5d0f898aca None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:41.328278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a6d24-5824-48eb-9119-5f5d0f898aca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:41.329728 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a6d24-5824-48eb-9119-5f5d0f898aca None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:41.348622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-7a8a6d24-5824-48eb-9119-5f5d0f898aca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:41.348622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:41.348622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:41.348622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:41.348622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:41.348622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:41.348622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:41.348622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:41.348622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:41.348622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:41.348622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:41.348622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:41.348622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:41.348622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:41.348622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:41.349574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:41.349574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:41.349574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:41.349574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:41.349574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:41.349574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:41.349574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:41.349574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:41.349574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:41.349574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:41.349574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:41.349574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:41.349574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:41.349574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:41.349574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:41.349574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:41.349574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:41.350604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:41.350604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:41.350604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:41.350604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:41.350604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:41.350604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:41.350604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:41.350604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:41.350604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 324/644] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:41.351940 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a6d24-5824-48eb-9119-5f5d0f898aca None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:41.364911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc91e5-110b-4d1a-8da1-41231d5dc758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:41.365718 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc91e5-110b-4d1a-8da1-41231d5dc758 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:41.365999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc91e5-110b-4d1a-8da1-41231d5dc758 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:41.366272 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc91e5-110b-4d1a-8da1-41231d5dc758 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:41.380429 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc91e5-110b-4d1a-8da1-41231d5dc758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:41.388220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-d5fc91e5-110b-4d1a-8da1-41231d5dc758 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:41.388220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:41.388220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:41.388220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:41.388220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:41.388220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:41.388220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:41.388220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:41.388220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:41.388220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:41.388220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:41.388220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:41.388220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:41.388220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:41.388220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:41.388220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:41.389331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:41.389331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:41.389331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:41.389331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:41.389331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:41.389331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:41.389331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:41.389331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:41.389331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:41.389331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:41.389331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:41.389331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:41.389331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:41.389331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:41.389331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:41.389331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:41.390476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:41.390476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:41.390476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:41.390476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:41.390476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:41.390476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:41.390476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:41.390476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:41.390476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:41.390476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:41.390476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:41.390476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:41.390476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 321/645] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:41.390476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc91e5-110b-4d1a-8da1-41231d5dc758 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:41.467985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:41.470952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a6d24-5824-48eb-9119-5f5d0f898aca None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:41.500918 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:41.504302 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc91e5-110b-4d1a-8da1-41231d5dc758 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:41.507922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90118804-e38e-4b82-b99c-44279048d675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:41.508590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90118804-e38e-4b82-b99c-44279048d675 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:41.508873 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90118804-e38e-4b82-b99c-44279048d675 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:41.509149 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90118804-e38e-4b82-b99c-44279048d675 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:41.509990 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90118804-e38e-4b82-b99c-44279048d675 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:41.510299 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90118804-e38e-4b82-b99c-44279048d675 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:41.511740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90118804-e38e-4b82-b99c-44279048d675 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:41.532975 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 325/646] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jun 22 11:32:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:41.533822 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90118804-e38e-4b82-b99c-44279048d675 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:41.552629 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b430abb-faf8-4a01-8fc7-2d65887589c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:41.553298 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b430abb-faf8-4a01-8fc7-2d65887589c0 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:41.553581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b430abb-faf8-4a01-8fc7-2d65887589c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:41.553877 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b430abb-faf8-4a01-8fc7-2d65887589c0 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:41.554628 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b430abb-faf8-4a01-8fc7-2d65887589c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:41.554932 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b430abb-faf8-4a01-8fc7-2d65887589c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:41.556372 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b430abb-faf8-4a01-8fc7-2d65887589c0 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:41.569857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 322/647] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:32:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:41.569857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b430abb-faf8-4a01-8fc7-2d65887589c0 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:41.651400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90118804-e38e-4b82-b99c-44279048d675 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:41.679983 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b430abb-faf8-4a01-8fc7-2d65887589c0 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:41.687629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33aaf5a-37a4-48e8-8895-3476b0cdd0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:41.688287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aaf5a-37a4-48e8-8895-3476b0cdd0a1 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:41.688568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aaf5a-37a4-48e8-8895-3476b0cdd0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:41.688939 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aaf5a-37a4-48e8-8895-3476b0cdd0a1 None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:41.689750 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33aaf5a-37a4-48e8-8895-3476b0cdd0a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:41.690055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33aaf5a-37a4-48e8-8895-3476b0cdd0a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:41.691544 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33aaf5a-37a4-48e8-8895-3476b0cdd0a1 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:41.703705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 326/648] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:41.709676 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33aaf5a-37a4-48e8-8895-3476b0cdd0a1 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:41.716523 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95180b7-b2a9-4661-8118-7ae164cdde11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:41.717288 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95180b7-b2a9-4661-8118-7ae164cdde11 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:41.717645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95180b7-b2a9-4661-8118-7ae164cdde11 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:41.718058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95180b7-b2a9-4661-8118-7ae164cdde11 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:41.718803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95180b7-b2a9-4661-8118-7ae164cdde11 None admin] RBAC: Adding query filter params (name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:41.719178 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95180b7-b2a9-4661-8118-7ae164cdde11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:41.720728 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95180b7-b2a9-4661-8118-7ae164cdde11 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:41.730844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 323/649] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:32:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:41.736122 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:41.736286 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95180b7-b2a9-4661-8118-7ae164cdde11 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:41.761618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:41.828013 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33aaf5a-37a4-48e8-8895-3476b0cdd0a1 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:41.847626 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95180b7-b2a9-4661-8118-7ae164cdde11 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:41.863653 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b1754-54cb-40cc-8d3f-d812c48c67b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:41.864392 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b1754-54cb-40cc-8d3f-d812c48c67b0 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:41.864745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b1754-54cb-40cc-8d3f-d812c48c67b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:41.865088 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b1754-54cb-40cc-8d3f-d812c48c67b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:41.884213 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fcbfc-e470-4b49-b548-4207fa776bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:41.885097 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fcbfc-e470-4b49-b548-4207fa776bcd None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:41.885450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fcbfc-e470-4b49-b548-4207fa776bcd None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:41.885797 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fcbfc-e470-4b49-b548-4207fa776bcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:41.898243 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fcbfc-e470-4b49-b548-4207fa776bcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:41.899840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fcbfc-e470-4b49-b548-4207fa776bcd None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:41.918855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-754fcbfc-e470-4b49-b548-4207fa776bcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:41.918855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:41.918855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:41.918855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:41.918855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:41.918855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:41.918855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:41.918855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:41.918855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:41.918855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:41.918855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:41.918855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:41.918855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:41.918855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:41.918855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:41.920120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:41.920120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:41.920120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:41.920120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:41.920120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:41.920120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:41.920120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:41.920120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:41.920120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:41.920120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:41.920120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:41.920120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:41.920120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:41.920120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:41.920120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:41.920120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:41.920120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:41.921242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:41.921242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:41.921242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:41.921242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:41.922005 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 324/650] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:41.922724 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fcbfc-e470-4b49-b548-4207fa776bcd None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:41.946907 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b1754-54cb-40cc-8d3f-d812c48c67b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:41.948387 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b1754-54cb-40cc-8d3f-d812c48c67b0 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:41.970321 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:42.007053 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:42.035015 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fcbfc-e470-4b49-b548-4207fa776bcd None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:42.067059 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 327/651] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:42.068632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b1754-54cb-40cc-8d3f-d812c48c67b0 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:42.080192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13af060-99c1-4548-9b26-e78b2c963f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:42.081697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13af060-99c1-4548-9b26-e78b2c963f1a None admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:42.082208 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13af060-99c1-4548-9b26-e78b2c963f1a None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:42.082682 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13af060-99c1-4548-9b26-e78b2c963f1a None admin] PATH_INFO: `/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/dcd642e290e44e67b56eb3b8361d7af3/roles/64c2d69f15ca458c8267c1acfd42d9fc` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:42.098994 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13af060-99c1-4548-9b26-e78b2c963f1a None admin] RBAC: Authorizing `identity:create_grant(project_id=47d2bb094c9843c9a7726f1d672d2433, user_id=dcd642e290e44e67b56eb3b8361d7af3, role_id=64c2d69f15ca458c8267c1acfd42d9fc)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:42.100753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13af060-99c1-4548-9b26-e78b2c963f1a None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:42.127377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 325/652] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:32:41 2022] PUT /identity/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/dcd642e290e44e67b56eb3b8361d7af3/roles/64c2d69f15ca458c8267c1acfd42d9fc => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:32:42.127377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13af060-99c1-4548-9b26-e78b2c963f1a None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:42.187985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b1754-54cb-40cc-8d3f-d812c48c67b0 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:42.213093 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:42.234493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467158f-6926-4549-9de7-0d44982b4356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:42.235668 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467158f-6926-4549-9de7-0d44982b4356 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:42.236207 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467158f-6926-4549-9de7-0d44982b4356 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:42.236628 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467158f-6926-4549-9de7-0d44982b4356 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:42.239563 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13af060-99c1-4548-9b26-e78b2c963f1a None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:42.249785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9467158f-6926-4549-9de7-0d44982b4356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:42.251718 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9467158f-6926-4549-9de7-0d44982b4356 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:42.267236 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:42.274440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-9467158f-6926-4549-9de7-0d44982b4356 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:42.274440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:42.274440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:42.274440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:42.274440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:42.274440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:42.274440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:42.274440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:42.274440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:42.274440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:42.274440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:42.274440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:42.274440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:42.274440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:42.274440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:42.275659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:42.275659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:42.275659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:42.275659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:42.275659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:42.275659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:42.275659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:42.275659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:42.275659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:42.275659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:42.275659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:42.275659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:42.275659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:42.275659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:42.275659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:42.275659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:42.275659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:42.276635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:42.276635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:42.276635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:42.276635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:42.279169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 328/653] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:42.280286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467158f-6926-4549-9de7-0d44982b4356 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:42.281464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17b8691-9212-4bc8-91db-665a1a878842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:42.282220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b8691-9212-4bc8-91db-665a1a878842 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:42.282729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b8691-9212-4bc8-91db-665a1a878842 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:42.283090 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b8691-9212-4bc8-91db-665a1a878842 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:42.283993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b8691-9212-4bc8-91db-665a1a878842 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:42.284482 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b8691-9212-4bc8-91db-665a1a878842 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:42.286373 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b8691-9212-4bc8-91db-665a1a878842 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:42.300597 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 326/654] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:42.301582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17b8691-9212-4bc8-91db-665a1a878842 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:42.405560 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467158f-6926-4549-9de7-0d44982b4356 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:42.433045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17b8691-9212-4bc8-91db-665a1a878842 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:42.442632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55d406-1c68-4fed-bedc-4ed8f4a2e96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:42.443843 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55d406-1c68-4fed-bedc-4ed8f4a2e96e None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:42.444315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55d406-1c68-4fed-bedc-4ed8f4a2e96e None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:42.444831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55d406-1c68-4fed-bedc-4ed8f4a2e96e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:42.457463 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe55d406-1c68-4fed-bedc-4ed8f4a2e96e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:42.459283 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe55d406-1c68-4fed-bedc-4ed8f4a2e96e None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:42.469821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b00350-53c7-4a21-84b9-9b67220ae6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:42.470487 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b00350-53c7-4a21-84b9-9b67220ae6ae None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:42.470768 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b00350-53c7-4a21-84b9-9b67220ae6ae None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:42.471043 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b00350-53c7-4a21-84b9-9b67220ae6ae None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:42.471793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b00350-53c7-4a21-84b9-9b67220ae6ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:42.472666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b00350-53c7-4a21-84b9-9b67220ae6ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:42.474053 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b00350-53c7-4a21-84b9-9b67220ae6ae None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:42.480225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-fe55d406-1c68-4fed-bedc-4ed8f4a2e96e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:42.480225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:42.480225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:42.480225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:42.480225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:42.480225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:42.480225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:42.480225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:42.480225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:42.480225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:42.480225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:42.480225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:42.480225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:42.480225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:42.480225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:42.481413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:42.481413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:42.481413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:42.481413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:42.481413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:42.481413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:42.481413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:42.481413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:42.481413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:42.481413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:42.481413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:42.481413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:42.481413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:42.481413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:42.481413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:42.481413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:42.481413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:42.482278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:42.482278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:42.482278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:42.482278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:42.484667 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 329/655] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:42.485734 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55d406-1c68-4fed-bedc-4ed8f4a2e96e None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:42.492297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 327/656] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:42.493241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b00350-53c7-4a21-84b9-9b67220ae6ae None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:42.513287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:42.547417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:42.599811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55d406-1c68-4fed-bedc-4ed8f4a2e96e None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:42.606347 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b00350-53c7-4a21-84b9-9b67220ae6ae None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:42.637528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532f5f0-3e40-46be-8f4a-a6b6a780257f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:42.638804 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532f5f0-3e40-46be-8f4a-a6b6a780257f None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:42.639398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532f5f0-3e40-46be-8f4a-a6b6a780257f None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:42.639895 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532f5f0-3e40-46be-8f4a-a6b6a780257f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:42.644471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502e237-b179-4744-baec-12ade85da7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:42.645136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502e237-b179-4744-baec-12ade85da7a4 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:42.645424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502e237-b179-4744-baec-12ade85da7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:42.645695 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502e237-b179-4744-baec-12ade85da7a4 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:42.646414 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502e237-b179-4744-baec-12ade85da7a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:42.646715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502e237-b179-4744-baec-12ade85da7a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:42.648111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502e237-b179-4744-baec-12ade85da7a4 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:42.650545 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532f5f0-3e40-46be-8f4a-a6b6a780257f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:42.652222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532f5f0-3e40-46be-8f4a-a6b6a780257f None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:42.662734 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 328/657] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:42.663618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502e237-b179-4744-baec-12ade85da7a4 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:42.672554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-0532f5f0-3e40-46be-8f4a-a6b6a780257f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 22 11:32:42.672554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:42.672554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:42.672554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:42.672554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:42.672554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:42.672554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:42.672554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:42.672554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:42.672554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:42.672554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:42.672554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:42.672554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:42.672554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:42.672554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:42.672554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:42.673514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:42.673514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:42.673514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:42.673514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:42.673514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:42.673514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:42.673514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:42.673514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:42.673514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:42.673514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:42.673514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:42.673514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:42.673514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:42.673514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:42.673514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:42.673514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:42.673514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:42.673514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:42.674433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:42.674433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:42.674433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:42.674433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:42.674433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:42.674433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:42.674433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:42.674433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:42.674433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 22 11:32:42.674433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:42.677055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 330/658] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:42.678049 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532f5f0-3e40-46be-8f4a-a6b6a780257f None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:42.764282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:42.777092 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502e237-b179-4744-baec-12ade85da7a4 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:42.779889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:42.787463 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532f5f0-3e40-46be-8f4a-a6b6a780257f None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:42.818634 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108823ac-edfe-4fb4-8f38-9de3aeb05f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:42.819826 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108823ac-edfe-4fb4-8f38-9de3aeb05f4c None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:42.820289 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108823ac-edfe-4fb4-8f38-9de3aeb05f4c None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:42.820737 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108823ac-edfe-4fb4-8f38-9de3aeb05f4c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:42.827137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984d3bd-6703-4887-a2d7-1bb1f82baaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:42.828034 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984d3bd-6703-4887-a2d7-1bb1f82baaa0 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:42.828385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984d3bd-6703-4887-a2d7-1bb1f82baaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:42.828730 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984d3bd-6703-4887-a2d7-1bb1f82baaa0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:42.830900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108823ac-edfe-4fb4-8f38-9de3aeb05f4c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:42.833506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108823ac-edfe-4fb4-8f38-9de3aeb05f4c None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:42.836815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984d3bd-6703-4887-a2d7-1bb1f82baaa0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:42.838292 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3984d3bd-6703-4887-a2d7-1bb1f82baaa0 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:42.849238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-108823ac-edfe-4fb4-8f38-9de3aeb05f4c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 22 11:32:42.849238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:42.849238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:42.849238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:42.849238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:42.849238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:42.849238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:42.849238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:42.849238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:42.849238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:42.849238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:42.849238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:42.849238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:42.849238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:42.849238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:42.849238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:42.850206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:42.850206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:42.850206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:42.850206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:42.850206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:42.850206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:42.850206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:42.850206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:42.850206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:42.850206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:42.850206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:42.850206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:42.850206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:42.850206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:42.850206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:42.850206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:42.850206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:42.850206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:42.851154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:42.851154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:42.851154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:42.851154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:42.851154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:42.851154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:42.851154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:42.851154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:42.851154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 22 11:32:42.851154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:42.853177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-3984d3bd-6703-4887-a2d7-1bb1f82baaa0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:42.853177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:42.853177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:42.853177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:42.853177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:42.853177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:42.853177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:42.853177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:42.853177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:42.853177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:42.853177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:42.853177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:42.853177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:42.853177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:42.853177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:42.854016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:42.854016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:42.854016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:42.854016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:42.854016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:42.854016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:42.854016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:42.854016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:42.854016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:42.854016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:42.854016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:42.854016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:42.854016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:42.854016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:42.854016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:42.854016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:42.854016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:42.854877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:42.854877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:42.854877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:42.854877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:42.854877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:42.854877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:42.854877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:42.854877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:42.860704 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 331/660] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:42.860704 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984d3bd-6703-4887-a2d7-1bb1f82baaa0 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:42.861026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 329/660] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:42.862084 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108823ac-edfe-4fb4-8f38-9de3aeb05f4c None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:42.969742 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984d3bd-6703-4887-a2d7-1bb1f82baaa0 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:42.991291 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108823ac-edfe-4fb4-8f38-9de3aeb05f4c None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:43.010028 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70eacaf-43ec-43d6-b7af-5a6da488ec0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:43.011125 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70eacaf-43ec-43d6-b7af-5a6da488ec0c None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:43.011635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70eacaf-43ec-43d6-b7af-5a6da488ec0c None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:43.011910 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70eacaf-43ec-43d6-b7af-5a6da488ec0c None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:43.012738 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70eacaf-43ec-43d6-b7af-5a6da488ec0c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:43.013044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70eacaf-43ec-43d6-b7af-5a6da488ec0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:43.014520 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70eacaf-43ec-43d6-b7af-5a6da488ec0c None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:43.027693 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba9852-b004-4dab-a5b2-81a973a932c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:43.028356 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9852-b004-4dab-a5b2-81a973a932c5 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:43.028637 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9852-b004-4dab-a5b2-81a973a932c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:43.028910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9852-b004-4dab-a5b2-81a973a932c5 None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:43.029354 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:43.029929 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba9852-b004-4dab-a5b2-81a973a932c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:43.030039 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba9852-b004-4dab-a5b2-81a973a932c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:43.031488 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba9852-b004-4dab-a5b2-81a973a932c5 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:43.034483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 332/661] 10.222.0.36 () {56 vars in 1137 bytes} [Wed Jun 22 11:32:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:43.036137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70eacaf-43ec-43d6-b7af-5a6da488ec0c None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:43.046196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 330/662] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:43.048148 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30a210-5776-4504-b438-219a091feced None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:43.048517 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30a210-5776-4504-b438-219a091feced None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:43.048876 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30a210-5776-4504-b438-219a091feced None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:43.051260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 331/663] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:43.052062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30a210-5776-4504-b438-219a091feced None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:43.078453 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:43.149542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70eacaf-43ec-43d6-b7af-5a6da488ec0c None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:43.173237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30a210-5776-4504-b438-219a091feced None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:43.189853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0d31f-13e4-473c-8450-522ad3ba4a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:43.190539 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0d31f-13e4-473c-8450-522ad3ba4a59 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:43.190853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0d31f-13e4-473c-8450-522ad3ba4a59 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:43.191129 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0d31f-13e4-473c-8450-522ad3ba4a59 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:43.192005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0d31f-13e4-473c-8450-522ad3ba4a59 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:43.192313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0d31f-13e4-473c-8450-522ad3ba4a59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:43.193809 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0d31f-13e4-473c-8450-522ad3ba4a59 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:43.215962 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 333/664] 10.222.0.36 () {56 vars in 1139 bytes} [Wed Jun 22 11:32:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:43.215962 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0d31f-13e4-473c-8450-522ad3ba4a59 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:43.216232 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa69942-632a-4071-a902-e2b7be8de45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:43.216232 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa69942-632a-4071-a902-e2b7be8de45e None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:43.216232 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa69942-632a-4071-a902-e2b7be8de45e None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:43.216232 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa69942-632a-4071-a902-e2b7be8de45e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:43.274413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:43.289555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa69942-632a-4071-a902-e2b7be8de45e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:43.291008 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa69942-632a-4071-a902-e2b7be8de45e None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:43.315263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:43.324943 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0d31f-13e4-473c-8450-522ad3ba4a59 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:43.363240 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05501d4d-3c88-450b-98b3-b5748bbbb4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:43.364171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05501d4d-3c88-450b-98b3-b5748bbbb4bf None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:43.364593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05501d4d-3c88-450b-98b3-b5748bbbb4bf None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:43.365018 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05501d4d-3c88-450b-98b3-b5748bbbb4bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:43.365852 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05501d4d-3c88-450b-98b3-b5748bbbb4bf None admin] RBAC: Adding query filter params (role.id=64c2d69f15ca458c8267c1acfd42d9fc, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=7ed133721ed044708ee362c93f564d07) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:43.366156 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05501d4d-3c88-450b-98b3-b5748bbbb4bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:43.367767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05501d4d-3c88-450b-98b3-b5748bbbb4bf None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:43.383377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 334/665] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:32:43 2022] GET /identity/v3/role_assignments?user.id=7ed133721ed044708ee362c93f564d07&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=64c2d69f15ca458c8267c1acfd42d9fc => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:43.383377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaec425-32e5-4a89-b065-4d32d6fc39f9 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:43.383377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaec425-32e5-4a89-b065-4d32d6fc39f9 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:43.383377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaec425-32e5-4a89-b065-4d32d6fc39f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:43.401064 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-aaaec425-32e5-4a89-b065-4d32d6fc39f9 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:43.402393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-aaaec425-32e5-4a89-b065-4d32d6fc39f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:43.425346 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 332/666] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:43.426439 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa69942-632a-4071-a902-e2b7be8de45e None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:43.497119 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 335/667] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:43.498106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaec425-32e5-4a89-b065-4d32d6fc39f9 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:43.542634 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa69942-632a-4071-a902-e2b7be8de45e None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:43.571949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:43.572140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:43.585780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e49ba-1bc6-441a-9e90-e6cbfe5ab3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:43.586579 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e49ba-1bc6-441a-9e90-e6cbfe5ab3a2 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:43.586983 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e49ba-1bc6-441a-9e90-e6cbfe5ab3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:43.587400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e49ba-1bc6-441a-9e90-e6cbfe5ab3a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:43.633559 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaec425-32e5-4a89-b065-4d32d6fc39f9 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:43.667290 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e49ba-1bc6-441a-9e90-e6cbfe5ab3a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:43.668796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e49ba-1bc6-441a-9e90-e6cbfe5ab3a2 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:43.684147 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d516a-baae-445d-baf1-d961a22f9164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:43.685428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d516a-baae-445d-baf1-d961a22f9164 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:43.685916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d516a-baae-445d-baf1-d961a22f9164 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:43.686416 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d516a-baae-445d-baf1-d961a22f9164 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:43.699209 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d516a-baae-445d-baf1-d961a22f9164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:43.700727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d516a-baae-445d-baf1-d961a22f9164 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:43.720861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-c68d516a-baae-445d-baf1-d961a22f9164 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:43.720861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:43.720861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:43.720861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:43.720861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:43.720861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:43.720861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:43.720861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:43.720861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:43.720861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:43.720861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:43.720861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:43.720861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:43.720861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:43.720861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:43.722138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:43.722138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:43.722138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:43.722138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:43.722138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:43.722138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:43.722138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:43.722138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:43.722138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:43.722138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:43.722138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:43.722138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:43.722138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:43.722138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:43.722138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:43.722138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:43.722138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:43.724732 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:43.724732 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:43.724732 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:43.724732 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:43.724732 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 336/668] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:43.724732 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d516a-baae-445d-baf1-d961a22f9164 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:43.806216 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 333/669] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:43.807553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e49ba-1bc6-441a-9e90-e6cbfe5ab3a2 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:43.833732 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d516a-baae-445d-baf1-d961a22f9164 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:43.837153 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:43.843377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:43.876453 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74d4dd-27fe-4b88-8d90-30e3bd05eae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:43.877357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74d4dd-27fe-4b88-8d90-30e3bd05eae0 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:43.877713 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74d4dd-27fe-4b88-8d90-30e3bd05eae0 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:43.878370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74d4dd-27fe-4b88-8d90-30e3bd05eae0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:43.893607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74d4dd-27fe-4b88-8d90-30e3bd05eae0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:43.902430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-cb74d4dd-27fe-4b88-8d90-30e3bd05eae0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:32:43.902430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:43.902430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:43.902430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:43.902430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:43.902430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:43.902430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:43.902430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:43.902430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:43.902430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:43.902430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:43.902430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:43.902430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:43.902430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:43.902430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:43.902430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:43.903429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:43.903429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:43.903429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:43.903429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:43.903429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:43.903429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:43.903429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:43.903429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:43.903429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:43.903429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:43.903429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:43.903429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:43.903429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:43.903429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:43.903429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:43.903429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:43.904480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:43.904480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:43.904480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:43.904480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:43.904480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:43.904480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:43.904480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:43.904480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:43.904480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:43.904480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:43.904480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:32:43.904480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:43.906864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 337/670] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:32:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:43.907873 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74d4dd-27fe-4b88-8d90-30e3bd05eae0 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:43.946489 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e49ba-1bc6-441a-9e90-e6cbfe5ab3a2 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:43.985081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccdc77-aa5d-4213-8b5a-25edd938edfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:43.985848 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccdc77-aa5d-4213-8b5a-25edd938edfe None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:43.986206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccdc77-aa5d-4213-8b5a-25edd938edfe None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:43.986555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ccdc77-aa5d-4213-8b5a-25edd938edfe None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:43.987487 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ccdc77-aa5d-4213-8b5a-25edd938edfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:43.988365 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ccdc77-aa5d-4213-8b5a-25edd938edfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:43.989839 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ccdc77-aa5d-4213-8b5a-25edd938edfe None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:44.003294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 334/671] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:44.004312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccdc77-aa5d-4213-8b5a-25edd938edfe None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:44.018626 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74d4dd-27fe-4b88-8d90-30e3bd05eae0 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:44.054986 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eedc05-ed9e-4470-b05d-66a6cdbfef94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:44.056061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eedc05-ed9e-4470-b05d-66a6cdbfef94 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:44.056577 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eedc05-ed9e-4470-b05d-66a6cdbfef94 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:44.057005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eedc05-ed9e-4470-b05d-66a6cdbfef94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:44.074244 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eedc05-ed9e-4470-b05d-66a6cdbfef94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:44.076832 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eedc05-ed9e-4470-b05d-66a6cdbfef94 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:44.100316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-69eedc05-ed9e-4470-b05d-66a6cdbfef94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:44.100316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:44.100316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:44.100316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:44.100316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:44.100316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:44.100316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:44.100316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:44.100316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:44.100316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:44.100316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:44.100316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:44.100316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:44.100316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:44.100316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:44.101510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:44.101510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:44.101510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:44.101510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:44.101510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:44.101510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:44.101510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:44.101510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:44.101510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:44.101510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:44.101510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:44.101510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:44.101510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:44.101510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:44.101510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:44.101510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:44.101510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:44.102631 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:44.102631 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:44.102631 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:44.102631 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:44.104282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 338/672] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:44.105429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eedc05-ed9e-4470-b05d-66a6cdbfef94 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:44.108915 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:44.129379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:44.135559 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ccdc77-aa5d-4213-8b5a-25edd938edfe None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:44.178270 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce74b7-d765-4379-8129-a8d64dcc1178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:44.179737 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce74b7-d765-4379-8129-a8d64dcc1178 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:44.180215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce74b7-d765-4379-8129-a8d64dcc1178 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:44.180958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce74b7-d765-4379-8129-a8d64dcc1178 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:44.182081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce74b7-d765-4379-8129-a8d64dcc1178 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:44.182580 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce74b7-d765-4379-8129-a8d64dcc1178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:44.184808 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce74b7-d765-4379-8129-a8d64dcc1178 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:44.197049 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 335/673] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:44.198162 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce74b7-d765-4379-8129-a8d64dcc1178 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:44.243574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eedc05-ed9e-4470-b05d-66a6cdbfef94 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:44.290596 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8147672-8757-4bff-8165-82e3561eb490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:44.292255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8147672-8757-4bff-8165-82e3561eb490 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:44.292749 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8147672-8757-4bff-8165-82e3561eb490 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:44.293262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8147672-8757-4bff-8165-82e3561eb490 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:44.303066 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8147672-8757-4bff-8165-82e3561eb490 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:44.304596 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8147672-8757-4bff-8165-82e3561eb490 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:44.319228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-e8147672-8757-4bff-8165-82e3561eb490 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:44.319228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:44.319228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:44.319228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:44.319228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:44.319228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:44.319228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:44.319228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:44.319228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:44.319228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:44.319228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:44.319228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:44.319228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:44.319228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:44.319228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:44.320198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:44.320198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:44.320198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:44.320198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:44.320198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:44.320198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:44.320198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:44.320198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:44.320198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:44.320198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:44.320198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:44.320198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:44.320198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:44.320198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:44.320198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:44.320198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:44.320198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:44.321094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:44.321094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:44.321094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:44.321094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:44.321094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:44.321094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:44.321094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:44.321094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:44.321094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 339/674] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:44.321601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8147672-8757-4bff-8165-82e3561eb490 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:44.327335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce74b7-d765-4379-8129-a8d64dcc1178 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:44.364702 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35ec77-cd54-4026-b03e-8ddb04fdec6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:44.365373 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35ec77-cd54-4026-b03e-8ddb04fdec6b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:44.365659 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35ec77-cd54-4026-b03e-8ddb04fdec6b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:44.365939 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35ec77-cd54-4026-b03e-8ddb04fdec6b None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:44.366669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35ec77-cd54-4026-b03e-8ddb04fdec6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:44.366973 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35ec77-cd54-4026-b03e-8ddb04fdec6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:44.368418 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35ec77-cd54-4026-b03e-8ddb04fdec6b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:44.384606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 336/675] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:44.384606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378348e3-849c-4699-914e-2581190803c4 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:44.384606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378348e3-849c-4699-914e-2581190803c4 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:44.384606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378348e3-849c-4699-914e-2581190803c4 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:44.384606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 337/676] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:44.384606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378348e3-849c-4699-914e-2581190803c4 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:44.393909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:44.412067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:44.439420 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8147672-8757-4bff-8165-82e3561eb490 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:44.474422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55c1fc-1a5e-418a-9271-98b61a164896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:44.475207 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55c1fc-1a5e-418a-9271-98b61a164896 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:44.475520 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55c1fc-1a5e-418a-9271-98b61a164896 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:44.475791 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55c1fc-1a5e-418a-9271-98b61a164896 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:44.485668 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe55c1fc-1a5e-418a-9271-98b61a164896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:44.487107 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe55c1fc-1a5e-418a-9271-98b61a164896 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:44.492595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378348e3-849c-4699-914e-2581190803c4 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:44.504553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-fe55c1fc-1a5e-418a-9271-98b61a164896 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:44.504553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:44.504553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:44.504553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:44.504553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:44.504553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:44.504553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:44.504553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:44.504553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:44.504553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:44.504553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:44.504553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:44.504553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:44.504553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:44.504553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:44.505970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:44.505970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:44.505970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:44.505970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:44.505970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:44.505970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:44.505970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:44.505970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:44.505970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:44.505970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:44.505970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:44.505970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:44.505970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:44.505970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:44.505970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:44.505970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:44.505970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:44.507027 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:44.507027 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:44.507027 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:44.507027 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:44.510496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 340/677] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:44.511229 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55c1fc-1a5e-418a-9271-98b61a164896 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:44.532421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e97cc-bfd3-4afc-83df-538e2e4c5128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:44.533083 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e97cc-bfd3-4afc-83df-538e2e4c5128 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:44.533370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e97cc-bfd3-4afc-83df-538e2e4c5128 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:44.533661 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e97cc-bfd3-4afc-83df-538e2e4c5128 None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:44.534493 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e97cc-bfd3-4afc-83df-538e2e4c5128 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:44.535320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e97cc-bfd3-4afc-83df-538e2e4c5128 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:44.536810 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e97cc-bfd3-4afc-83df-538e2e4c5128 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:44.549453 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 338/678] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:44.550666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224de1c-da67-4619-a64e-cdf1e5f95f2e None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:44.550952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224de1c-da67-4619-a64e-cdf1e5f95f2e None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:44.551228 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224de1c-da67-4619-a64e-cdf1e5f95f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:44.567184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-7224de1c-da67-4619-a64e-cdf1e5f95f2e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:44.568412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-7224de1c-da67-4619-a64e-cdf1e5f95f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:44.628389 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55c1fc-1a5e-418a-9271-98b61a164896 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:44.631543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:44.639717 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 339/679] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:44.640867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224de1c-da67-4619-a64e-cdf1e5f95f2e None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:44.666754 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:44.675917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9ce88-72f6-4790-8fa9-f9fb4e3c4819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:44.676712 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9ce88-72f6-4790-8fa9-f9fb4e3c4819 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:44.676993 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9ce88-72f6-4790-8fa9-f9fb4e3c4819 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:44.677268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9ce88-72f6-4790-8fa9-f9fb4e3c4819 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:44.684262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9ce88-72f6-4790-8fa9-f9fb4e3c4819 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:44.685698 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9ce88-72f6-4790-8fa9-f9fb4e3c4819 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:44.700609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-d9f9ce88-72f6-4790-8fa9-f9fb4e3c4819 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:44.700609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:44.700609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:44.700609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:44.700609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:44.700609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:44.700609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:44.700609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:44.700609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:44.700609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:44.700609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:44.700609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:44.700609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:44.700609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:44.700609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:44.701751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:44.701751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:44.701751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:44.701751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:44.701751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:44.701751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:44.701751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:44.701751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:44.701751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:44.701751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:44.701751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:44.701751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:44.701751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:44.701751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:44.701751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:44.701751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:44.701751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:44.702900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:44.702900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:44.702900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:44.702900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:44.702900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:44.702900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:44.702900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:44.702900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:44.707569 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 341/680] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:44.708460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9ce88-72f6-4790-8fa9-f9fb4e3c4819 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:44.763715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224de1c-da67-4619-a64e-cdf1e5f95f2e None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:44.802583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794f56f-0efe-481c-a610-f07c4cf6e990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:44.803430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794f56f-0efe-481c-a610-f07c4cf6e990 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:44.803847 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794f56f-0efe-481c-a610-f07c4cf6e990 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:44.804342 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794f56f-0efe-481c-a610-f07c4cf6e990 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:44.805269 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794f56f-0efe-481c-a610-f07c4cf6e990 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:44.805667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794f56f-0efe-481c-a610-f07c4cf6e990 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:44.807583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794f56f-0efe-481c-a610-f07c4cf6e990 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:44.813874 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9ce88-72f6-4790-8fa9-f9fb4e3c4819 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:44.824742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 340/681] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:44.824742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794f56f-0efe-481c-a610-f07c4cf6e990 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:44.852268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e6f8e-4448-4990-9432-500f6d299579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:44.853015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e6f8e-4448-4990-9432-500f6d299579 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:44.853366 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e6f8e-4448-4990-9432-500f6d299579 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:44.853708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e6f8e-4448-4990-9432-500f6d299579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:44.854613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e6f8e-4448-4990-9432-500f6d299579 None admin] RBAC: Adding query filter params (role.id=64c2d69f15ca458c8267c1acfd42d9fc, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=ceb170ac6353495b9d1dc6723d59567a) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:44.854999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e6f8e-4448-4990-9432-500f6d299579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:44.856468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e6f8e-4448-4990-9432-500f6d299579 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:44.862750 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 342/682] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:32:44 2022] GET /identity/v3/role_assignments?user.id=ceb170ac6353495b9d1dc6723d59567a&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=64c2d69f15ca458c8267c1acfd42d9fc => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:44.863662 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e6f8e-4448-4990-9432-500f6d299579 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:44.893529 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:44.957348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794f56f-0efe-481c-a610-f07c4cf6e990 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:44.963254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:44.997887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677f3e9-b75c-4f65-b664-20878a6ac83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:44.998735 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677f3e9-b75c-4f65-b664-20878a6ac83b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:44.999017 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677f3e9-b75c-4f65-b664-20878a6ac83b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:44.999292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677f3e9-b75c-4f65-b664-20878a6ac83b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:45.019931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677f3e9-b75c-4f65-b664-20878a6ac83b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:45.031776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-9677f3e9-b75c-4f65-b664-20878a6ac83b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:45.031776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:45.031776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:45.031776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:45.031776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:45.031776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:45.031776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:45.031776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:45.031776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:45.031776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:45.031776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:45.031776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:45.031776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:45.031776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:45.031776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:45.031776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:45.032911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:45.032911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:45.032911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:45.032911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:45.032911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:45.032911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:45.032911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:45.032911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:45.032911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:45.032911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:45.032911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:45.032911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:45.032911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:45.032911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:45.032911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:45.032911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:45.033774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:45.033774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:45.033774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:45.033774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:45.033774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:45.033774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:45.033774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:45.033774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:45.033774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:45.033774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:45.033774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:45.033774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:45.033774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 341/683] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:45.035285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677f3e9-b75c-4f65-b664-20878a6ac83b None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:45.036351 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e6f8e-4448-4990-9432-500f6d299579 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:45.089319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40de386-3209-4234-8ad8-f69c1b44e366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:45.090254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40de386-3209-4234-8ad8-f69c1b44e366 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:45.090545 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40de386-3209-4234-8ad8-f69c1b44e366 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:45.090829 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40de386-3209-4234-8ad8-f69c1b44e366 None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:45.091929 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40de386-3209-4234-8ad8-f69c1b44e366 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:45.092253 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40de386-3209-4234-8ad8-f69c1b44e366 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:45.094129 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40de386-3209-4234-8ad8-f69c1b44e366 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:45.109969 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 343/684] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:45.111615 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40de386-3209-4234-8ad8-f69c1b44e366 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:45.167435 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677f3e9-b75c-4f65-b664-20878a6ac83b None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:45.189430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:45.214201 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b70bfa-8860-494a-acd4-23ff773dc4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:45.216131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b70bfa-8860-494a-acd4-23ff773dc4bf None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:45.216643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b70bfa-8860-494a-acd4-23ff773dc4bf None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:45.217371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b70bfa-8860-494a-acd4-23ff773dc4bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:45.228233 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b70bfa-8860-494a-acd4-23ff773dc4bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:45.230678 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b70bfa-8860-494a-acd4-23ff773dc4bf None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:45.235580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40de386-3209-4234-8ad8-f69c1b44e366 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:45.250712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-f6b70bfa-8860-494a-acd4-23ff773dc4bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 22 11:32:45.250712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:45.250712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:45.250712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:45.250712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:45.250712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:45.250712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:45.250712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:45.250712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:45.250712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:45.250712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:45.250712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:45.250712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:45.250712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:45.250712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:45.250712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:45.251742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:45.251742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:45.251742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:45.251742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:45.251742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:45.251742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:45.251742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:45.251742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:45.251742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:45.251742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:45.251742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:45.251742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:45.251742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:45.251742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:45.251742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:45.251742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:45.251742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:45.251742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:45.252680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:45.252680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:45.252680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:45.252680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:45.252680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:45.252680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:45.252680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:45.252680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:45.252680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 22 11:32:45.252680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:45.254113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 342/685] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jun 22 11:32:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:45.254113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b70bfa-8860-494a-acd4-23ff773dc4bf None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:45.255957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:45.283806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452bae2-00c9-4280-80b8-c18ce1fb2271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:45.284986 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452bae2-00c9-4280-80b8-c18ce1fb2271 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:45.285470 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452bae2-00c9-4280-80b8-c18ce1fb2271 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:45.285985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452bae2-00c9-4280-80b8-c18ce1fb2271 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:45.287129 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452bae2-00c9-4280-80b8-c18ce1fb2271 None admin] RBAC: Adding query filter params (name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:45.287682 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452bae2-00c9-4280-80b8-c18ce1fb2271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:45.289412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452bae2-00c9-4280-80b8-c18ce1fb2271 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:45.299437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 344/686] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:32:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:45.301233 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452bae2-00c9-4280-80b8-c18ce1fb2271 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:45.363671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b70bfa-8860-494a-acd4-23ff773dc4bf None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:45.399793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d2e03-5d71-40bc-9dab-6828c927b097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:45.400487 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d2e03-5d71-40bc-9dab-6828c927b097 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:45.400774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d2e03-5d71-40bc-9dab-6828c927b097 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:45.401057 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d2e03-5d71-40bc-9dab-6828c927b097 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:45.401900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d2e03-5d71-40bc-9dab-6828c927b097 None admin] RBAC: Adding query filter params (role.id=64c2d69f15ca458c8267c1acfd42d9fc, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=c00d0bb777a044a3a4d4540d187d938a) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:45.402217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d2e03-5d71-40bc-9dab-6828c927b097 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:45.403842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d2e03-5d71-40bc-9dab-6828c927b097 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:45.410816 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 343/687] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:32:45 2022] GET /identity/v3/role_assignments?user.id=c00d0bb777a044a3a4d4540d187d938a&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=64c2d69f15ca458c8267c1acfd42d9fc => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:45.411727 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d2e03-5d71-40bc-9dab-6828c927b097 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:45.429942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452bae2-00c9-4280-80b8-c18ce1fb2271 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:45.470299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:45.516430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a332a-3188-4026-93c7-d590c7f320ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:45.517135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a332a-3188-4026-93c7-d590c7f320ce None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:45.517400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a332a-3188-4026-93c7-d590c7f320ce None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:45.517678 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a332a-3188-4026-93c7-d590c7f320ce None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:45.518552 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a332a-3188-4026-93c7-d590c7f320ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:45.519423 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a332a-3188-4026-93c7-d590c7f320ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:45.520936 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a332a-3188-4026-93c7-d590c7f320ce None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:45.545664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 345/688] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jun 22 11:32:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:45.551078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a332a-3188-4026-93c7-d590c7f320ce None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:45.581471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d2e03-5d71-40bc-9dab-6828c927b097 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:45.602207 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:45.650420 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b61d3-84f3-4c70-9328-325549fc3aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:45.651617 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b61d3-84f3-4c70-9328-325549fc3aa6 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:45.652112 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b61d3-84f3-4c70-9328-325549fc3aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:45.652614 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b61d3-84f3-4c70-9328-325549fc3aa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:45.664317 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b61d3-84f3-4c70-9328-325549fc3aa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:45.666108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b61d3-84f3-4c70-9328-325549fc3aa6 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:45.678096 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a332a-3188-4026-93c7-d590c7f320ce None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:45.684348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-e22b61d3-84f3-4c70-9328-325549fc3aa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:45.684348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:45.684348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:45.684348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:45.684348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:45.684348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:45.684348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:45.684348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:45.684348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:45.684348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:45.684348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:45.684348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:45.684348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:45.684348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:45.684348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:45.685249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:45.685249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:45.685249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:45.685249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:45.685249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:45.685249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:45.685249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:45.685249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:45.685249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:45.685249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:45.685249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:45.685249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:45.685249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:45.685249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:45.685249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:45.685249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:45.685249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:45.686156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:45.686156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:45.686156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:45.686156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:45.688423 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 344/689] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:45.694017 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b61d3-84f3-4c70-9328-325549fc3aa6 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:45.723668 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf766b2-258b-458d-8574-2ba939c39dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:45.724362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf766b2-258b-458d-8574-2ba939c39dca None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:45.724638 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf766b2-258b-458d-8574-2ba939c39dca None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:45.724911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf766b2-258b-458d-8574-2ba939c39dca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:45.816977 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf766b2-258b-458d-8574-2ba939c39dca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:45.819338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf766b2-258b-458d-8574-2ba939c39dca None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:45.823974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b61d3-84f3-4c70-9328-325549fc3aa6 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:45.829371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:45.859722 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:45.866627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b120f-0978-47c1-8718-5e96de18587f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:45.867575 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b120f-0978-47c1-8718-5e96de18587f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:45.868056 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b120f-0978-47c1-8718-5e96de18587f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:45.868508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b120f-0978-47c1-8718-5e96de18587f None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:45.869399 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b120f-0978-47c1-8718-5e96de18587f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:45.869877 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b120f-0978-47c1-8718-5e96de18587f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:45.871463 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b120f-0978-47c1-8718-5e96de18587f None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:45.892911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 345/690] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:45.895246 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e44d2-773e-4b33-a231-fc634d1dacf2 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:45.895788 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e44d2-773e-4b33-a231-fc634d1dacf2 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:45.896262 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e44d2-773e-4b33-a231-fc634d1dacf2 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:45.898013 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 346/691] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:45.902723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e44d2-773e-4b33-a231-fc634d1dacf2 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:45.952601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 346/692] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 22 11:32:45.954147 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75cab9-74db-4fc4-b014-7f6439d039b6 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:45.954439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75cab9-74db-4fc4-b014-7f6439d039b6 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:45.954708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75cab9-74db-4fc4-b014-7f6439d039b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:45.975311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-af75cab9-74db-4fc4-b014-7f6439d039b6 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:45.976792 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-af75cab9-74db-4fc4-b014-7f6439d039b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:46.029389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e44d2-773e-4b33-a231-fc634d1dacf2 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:46.058046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 347/693] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:46.058916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75cab9-74db-4fc4-b014-7f6439d039b6 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:46.074938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85b588-16ed-4f7e-9de8-fa6729071e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:46.076011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85b588-16ed-4f7e-9de8-fa6729071e91 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:46.076293 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85b588-16ed-4f7e-9de8-fa6729071e91 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:46.076570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85b588-16ed-4f7e-9de8-fa6729071e91 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:46.087325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85b588-16ed-4f7e-9de8-fa6729071e91 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:46.089461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85b588-16ed-4f7e-9de8-fa6729071e91 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:46.109320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-9d85b588-16ed-4f7e-9de8-fa6729071e91 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 22 11:32:46.109320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:46.109320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:46.109320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:46.109320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:46.109320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:46.109320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:46.109320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:46.109320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:46.109320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:46.109320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:46.109320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:46.109320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:46.109320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:46.109320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:46.109320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:46.110297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:46.110297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:46.110297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:46.110297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:46.110297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:46.110297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:46.110297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:46.110297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:46.110297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:46.110297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:46.110297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:46.110297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:46.110297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:46.110297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:46.110297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:46.110297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:46.110297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:46.110297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:46.111250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:46.111250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:46.111250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:46.111250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:46.111250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:46.111250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:46.111250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:46.111250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:46.111250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 22 11:32:46.111250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:46.111250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 347/694] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:32:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:46.111877 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85b588-16ed-4f7e-9de8-fa6729071e91 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:46.132470 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:46.133477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:46.186093 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75cab9-74db-4fc4-b014-7f6439d039b6 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:46.225163 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9c6f0-5448-495c-b96b-c31cf37caf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:46.226042 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9c6f0-5448-495c-b96b-c31cf37caf01 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:46.226450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9c6f0-5448-495c-b96b-c31cf37caf01 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:46.227099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9c6f0-5448-495c-b96b-c31cf37caf01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:46.239871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec9c6f0-5448-495c-b96b-c31cf37caf01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:46.241606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85b588-16ed-4f7e-9de8-fa6729071e91 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:46.241920 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec9c6f0-5448-495c-b96b-c31cf37caf01 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:46.262044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-dec9c6f0-5448-495c-b96b-c31cf37caf01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:46.262044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:46.262044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:46.262044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:46.262044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:46.262044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:46.262044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:46.262044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:46.262044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:46.262044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:46.262044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:46.262044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:46.262044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:46.262044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:46.262044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:46.263178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:46.263178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:46.263178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:46.263178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:46.263178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:46.263178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:46.263178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:46.263178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:46.263178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:46.263178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:46.263178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:46.263178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:46.263178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:46.263178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:46.263178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:46.263178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:46.263178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:46.264279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:46.264279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:46.264279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:46.264279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:46.270437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 348/695] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:46.270437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9c6f0-5448-495c-b96b-c31cf37caf01 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:46.292318 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cdade-8ca3-40c5-ba55-56c0b200bddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:46.293276 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cdade-8ca3-40c5-ba55-56c0b200bddb None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:46.293991 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cdade-8ca3-40c5-ba55-56c0b200bddb None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:46.294110 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015cdade-8ca3-40c5-ba55-56c0b200bddb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:46.306943 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015cdade-8ca3-40c5-ba55-56c0b200bddb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:46.308695 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015cdade-8ca3-40c5-ba55-56c0b200bddb None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:46.328253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-015cdade-8ca3-40c5-ba55-56c0b200bddb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:46.328253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:46.328253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:46.328253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:46.328253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:46.328253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:46.328253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:46.328253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:46.328253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:46.328253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:46.328253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:46.328253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:46.328253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:46.328253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:46.328253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:46.329343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:46.329343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:46.329343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:46.329343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:46.329343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:46.329343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:46.329343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:46.329343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:46.329343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:46.329343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:46.329343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:46.329343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:46.329343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:46.329343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:46.329343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:46.329343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:46.329343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:46.330434 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:46.330434 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:46.330434 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:46.330434 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:46.332207 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 348/696] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:46.333185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cdade-8ca3-40c5-ba55-56c0b200bddb None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:46.396925 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9c6f0-5448-495c-b96b-c31cf37caf01 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:46.404286 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:46.418923 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:46.442428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91c572-e999-437c-ae47-f65475fc2ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:46.443721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91c572-e999-437c-ae47-f65475fc2ff8 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:46.444213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91c572-e999-437c-ae47-f65475fc2ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:46.444720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91c572-e999-437c-ae47-f65475fc2ff8 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:46.445767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91c572-e999-437c-ae47-f65475fc2ff8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:46.446279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91c572-e999-437c-ae47-f65475fc2ff8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:46.447961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91c572-e999-437c-ae47-f65475fc2ff8 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:46.448692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015cdade-8ca3-40c5-ba55-56c0b200bddb None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:46.468743 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 349/697] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jun 22 11:32:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:46.470016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91c572-e999-437c-ae47-f65475fc2ff8 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:46.492374 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed551b-04cf-46ce-b38f-bc3b5a72c340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:46.492955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed551b-04cf-46ce-b38f-bc3b5a72c340 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:46.493234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed551b-04cf-46ce-b38f-bc3b5a72c340 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:46.493508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed551b-04cf-46ce-b38f-bc3b5a72c340 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:46.494230 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efed551b-04cf-46ce-b38f-bc3b5a72c340 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:46.494538 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efed551b-04cf-46ce-b38f-bc3b5a72c340 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:46.495946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efed551b-04cf-46ce-b38f-bc3b5a72c340 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:46.512920 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 349/698] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:46.514121 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed551b-04cf-46ce-b38f-bc3b5a72c340 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:46.592492 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91c572-e999-437c-ae47-f65475fc2ff8 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:46.619638 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed551b-04cf-46ce-b38f-bc3b5a72c340 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:46.633556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2a6a8-a775-40df-b3f4-7d4acf635ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:46.634453 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2a6a8-a775-40df-b3f4-7d4acf635ec9 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:46.634918 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2a6a8-a775-40df-b3f4-7d4acf635ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:46.635335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2a6a8-a775-40df-b3f4-7d4acf635ec9 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:46.636224 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2a6a8-a775-40df-b3f4-7d4acf635ec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:46.636679 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2a6a8-a775-40df-b3f4-7d4acf635ec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:46.638180 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2a6a8-a775-40df-b3f4-7d4acf635ec9 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:46.645823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:46.651833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 350/699] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:46.652764 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2a6a8-a775-40df-b3f4-7d4acf635ec9 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:46.660345 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac63d9-5b26-4f18-a680-cfa5c8663740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:46.661276 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac63d9-5b26-4f18-a680-cfa5c8663740 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:46.661798 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac63d9-5b26-4f18-a680-cfa5c8663740 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:46.662274 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac63d9-5b26-4f18-a680-cfa5c8663740 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:46.687194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:46.734558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac63d9-5b26-4f18-a680-cfa5c8663740 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:46.736297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac63d9-5b26-4f18-a680-cfa5c8663740 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:46.766671 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2a6a8-a775-40df-b3f4-7d4acf635ec9 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:46.804172 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee60ba-77a5-4982-afe3-956934295481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:46.805219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee60ba-77a5-4982-afe3-956934295481 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:46.805683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee60ba-77a5-4982-afe3-956934295481 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:46.806104 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee60ba-77a5-4982-afe3-956934295481 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:46.816409 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee60ba-77a5-4982-afe3-956934295481 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:46.818112 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee60ba-77a5-4982-afe3-956934295481 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:46.833686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-9fee60ba-77a5-4982-afe3-956934295481 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:46.833686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:46.833686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:46.833686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:46.833686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:46.833686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:46.833686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:46.833686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:46.833686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:46.833686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:46.833686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:46.833686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:46.833686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:46.833686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:46.833686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:46.834798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:46.834798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:46.834798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:46.834798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:46.834798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:46.834798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:46.834798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:46.834798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:46.834798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:46.834798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:46.834798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:46.834798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:46.834798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:46.834798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:46.834798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:46.834798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:46.834798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:46.835837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:46.835837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:46.835837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:46.835837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:46.835837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:46.835837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:46.835837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:46.835837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:46.837948 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 351/700] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:46.838998 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee60ba-77a5-4982-afe3-956934295481 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:46.864452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 350/701] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:46.865450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac63d9-5b26-4f18-a680-cfa5c8663740 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:46.890299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:46.939528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:46.953177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee60ba-77a5-4982-afe3-956934295481 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:46.974709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac63d9-5b26-4f18-a680-cfa5c8663740 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:46.988992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29d928-a855-425d-8469-f75f8ff06253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:46.989666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29d928-a855-425d-8469-f75f8ff06253 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:46.989950 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29d928-a855-425d-8469-f75f8ff06253 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:46.990229 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29d928-a855-425d-8469-f75f8ff06253 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:46.990989 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29d928-a855-425d-8469-f75f8ff06253 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:46.992380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29d928-a855-425d-8469-f75f8ff06253 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:47.015587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9206b54-e3c3-4d6b-a900-af83dffe5407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:47.016285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 352/702] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jun 22 11:32:46 2022] POST /identity/v3/users => generated 254 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:32:47.016914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9206b54-e3c3-4d6b-a900-af83dffe5407 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:47.017275 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29d928-a855-425d-8469-f75f8ff06253 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:47.017602 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9206b54-e3c3-4d6b-a900-af83dffe5407 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:47.018126 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9206b54-e3c3-4d6b-a900-af83dffe5407 None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:47.019186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9206b54-e3c3-4d6b-a900-af83dffe5407 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:47.019743 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9206b54-e3c3-4d6b-a900-af83dffe5407 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:47.022982 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9206b54-e3c3-4d6b-a900-af83dffe5407 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:47.039414 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 351/703] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:47.045808 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9206b54-e3c3-4d6b-a900-af83dffe5407 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:47.165477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:47.174269 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29d928-a855-425d-8469-f75f8ff06253 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:47.194130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9206b54-e3c3-4d6b-a900-af83dffe5407 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:47.226741 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e64a18-9de2-4d70-9635-868e4a55b448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:47.228233 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e64a18-9de2-4d70-9635-868e4a55b448 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:47.228233 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e64a18-9de2-4d70-9635-868e4a55b448 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:47.228233 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e64a18-9de2-4d70-9635-868e4a55b448 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:47.243394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e64a18-9de2-4d70-9635-868e4a55b448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:47.245676 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e64a18-9de2-4d70-9635-868e4a55b448 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:47.247151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe1866-bcc8-4ca6-bf32-e512a85bdb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:47.248236 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe1866-bcc8-4ca6-bf32-e512a85bdb55 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:47.248730 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe1866-bcc8-4ca6-bf32-e512a85bdb55 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:47.249232 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe1866-bcc8-4ca6-bf32-e512a85bdb55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:47.250940 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe1866-bcc8-4ca6-bf32-e512a85bdb55 None admin] RBAC: Adding query filter params (role.id=96c584428d924fdba9671949f760c406, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=479a376dcac14c509c1c55f0046d88cf) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:47.251539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe1866-bcc8-4ca6-bf32-e512a85bdb55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:47.254116 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe1866-bcc8-4ca6-bf32-e512a85bdb55 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:47.257300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:47.261397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 352/704] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:32:47 2022] GET /identity/v3/role_assignments?user.id=479a376dcac14c509c1c55f0046d88cf&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=96c584428d924fdba9671949f760c406 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:47.263098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5a4ce-b50f-4c9c-b33e-c9d813d5defa None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:47.263678 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5a4ce-b50f-4c9c-b33e-c9d813d5defa None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:47.264165 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5a4ce-b50f-4c9c-b33e-c9d813d5defa None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:47.270448 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 353/705] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:47.272497 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534637f-8e10-453d-a18f-bf2907b380eb None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:47.272752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534637f-8e10-453d-a18f-bf2907b380eb None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:47.273116 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534637f-8e10-453d-a18f-bf2907b380eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:47.274686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-46e64a18-9de2-4d70-9635-868e4a55b448 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:47.274686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:47.274686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:47.274686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:47.274686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:47.274686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:47.274686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:47.274686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:47.274686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:47.274686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:47.274686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:47.274686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:47.274686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:47.274686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:47.274686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:47.276687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:47.276687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:47.276687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:47.276687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:47.276687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:47.276687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:47.276687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:47.276687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:47.276687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:47.276687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:47.276687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:47.276687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:47.276687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:47.276687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:47.276687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:47.276687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:47.276687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:47.277688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:47.277688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:47.277688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:47.277688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:47.279848 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 353/706] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:47.286005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e64a18-9de2-4d70-9635-868e4a55b448 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:47.296751 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f534637f-8e10-453d-a18f-bf2907b380eb None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:47.297963 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f534637f-8e10-453d-a18f-bf2907b380eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:47.387391 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 354/707] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 22 11:32:47.423837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e64a18-9de2-4d70-9635-868e4a55b448 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:47.444274 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534637f-8e10-453d-a18f-bf2907b380eb None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:47.480356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a293f-4508-4b02-9756-2167765c3adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:47.481032 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a293f-4508-4b02-9756-2167765c3adf None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:47.481319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a293f-4508-4b02-9756-2167765c3adf None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:47.481601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a293f-4508-4b02-9756-2167765c3adf None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:47.482331 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a293f-4508-4b02-9756-2167765c3adf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:47.482645 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a293f-4508-4b02-9756-2167765c3adf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:47.484060 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a293f-4508-4b02-9756-2167765c3adf None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:47.496992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 354/708] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:47.505875 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a293f-4508-4b02-9756-2167765c3adf None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:47.545124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:47.568805 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534637f-8e10-453d-a18f-bf2907b380eb None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:47.572548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:47.618863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a293f-4508-4b02-9756-2167765c3adf None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:47.641677 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47783e8-e22a-4573-b677-1b692c442f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:47.642857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47783e8-e22a-4573-b677-1b692c442f22 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:47.643400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47783e8-e22a-4573-b677-1b692c442f22 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:47.643920 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47783e8-e22a-4573-b677-1b692c442f22 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:47.644788 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47783e8-e22a-4573-b677-1b692c442f22 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:47.651813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47783e8-e22a-4573-b677-1b692c442f22 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:47.657447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abc635-903a-4aa4-8fbd-eee75cf37312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:47.657447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abc635-903a-4aa4-8fbd-eee75cf37312 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:47.657447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abc635-903a-4aa4-8fbd-eee75cf37312 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:47.657447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abc635-903a-4aa4-8fbd-eee75cf37312 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:47.665139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71abc635-903a-4aa4-8fbd-eee75cf37312 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:47.666593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71abc635-903a-4aa4-8fbd-eee75cf37312 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:47.681164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-71abc635-903a-4aa4-8fbd-eee75cf37312 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:47.681164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:47.681164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:47.681164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:47.681164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:47.681164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:47.681164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:47.681164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:47.681164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:47.681164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:47.681164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:47.681164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:47.681164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:47.681164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:47.681164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:47.682055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:47.682055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:47.682055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:47.682055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:47.682055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:47.682055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:47.682055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:47.682055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:47.682055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:47.682055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:47.682055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:47.682055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:47.682055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:47.682055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:47.682055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:47.682055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:47.682055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:47.683375 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:47.683375 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:47.683375 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:47.683375 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:47.683375 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:47.683375 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:47.683375 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:47.683375 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:47.683375 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 355/709] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:47.684576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-e47783e8-e22a-4573-b677-1b692c442f22 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 22 11:32:47.684576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:47.684576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:32:47.684576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:32:47.684576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:32:47.684576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:32:47.684576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:32:47.684576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:32:47.684576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 22 11:32:47.684576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:47.684576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:32:47.684576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:47.684576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:47.684576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:47.684576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:47.684576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:47.685486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:47.685486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:47.685486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:47.685486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:47.685486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:47.685486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:47.685486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:47.685486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:32:47.685486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:32:47.685486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:32:47.685486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:32:47.685486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:47.685486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:47.685486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:47.685486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:47.685486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:47.686325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:47.686325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:47.686325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:47.686325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:47.686325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:47.686325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:47.686325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:47.686325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:47.686325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:47.686325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:47.686325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:47.686325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:32:47.686325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:32:47.686325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 22 11:32:47.686325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:47.688149 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abc635-903a-4aa4-8fbd-eee75cf37312 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:47.693677 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 355/710] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Jun 22 11:32:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:47.701321 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47783e8-e22a-4573-b677-1b692c442f22 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:47.808744 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abc635-903a-4aa4-8fbd-eee75cf37312 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:47.832850 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:47.845015 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47783e8-e22a-4573-b677-1b692c442f22 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:47.847751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407abfa5-df80-41b0-9815-e963a441ecd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:47.848424 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407abfa5-df80-41b0-9815-e963a441ecd6 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:47.848705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407abfa5-df80-41b0-9815-e963a441ecd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:47.848979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407abfa5-df80-41b0-9815-e963a441ecd6 None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:47.849765 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407abfa5-df80-41b0-9815-e963a441ecd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:47.850071 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407abfa5-df80-41b0-9815-e963a441ecd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:47.851526 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407abfa5-df80-41b0-9815-e963a441ecd6 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:47.865205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 356/711] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:47.866583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a04b5d-cfbd-4de9-a3f9-3477ae2abad1 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:47.866869 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a04b5d-cfbd-4de9-a3f9-3477ae2abad1 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:47.867143 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a04b5d-cfbd-4de9-a3f9-3477ae2abad1 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:47.868742 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 357/712] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:47.873094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a04b5d-cfbd-4de9-a3f9-3477ae2abad1 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:47.893125 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb2ff4-97ca-469b-9cb4-8c6ccf8ab2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:47.894175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb2ff4-97ca-469b-9cb4-8c6ccf8ab2ab None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:47.894463 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb2ff4-97ca-469b-9cb4-8c6ccf8ab2ab None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:47.894745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb2ff4-97ca-469b-9cb4-8c6ccf8ab2ab None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:47.895675 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb2ff4-97ca-469b-9cb4-8c6ccf8ab2ab None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:47.895986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb2ff4-97ca-469b-9cb4-8c6ccf8ab2ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:47.898107 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb2ff4-97ca-469b-9cb4-8c6ccf8ab2ab None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:47.904027 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 356/713] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jun 22 11:32:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:47.905450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2dc41-d909-4277-b0da-962b4891b2a6 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:47.905740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2dc41-d909-4277-b0da-962b4891b2a6 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:47.906062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2dc41-d909-4277-b0da-962b4891b2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:47.923697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:47.926528 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-feb2dc41-d909-4277-b0da-962b4891b2a6 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:47.928678 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-feb2dc41-d909-4277-b0da-962b4891b2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:48.005875 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a04b5d-cfbd-4de9-a3f9-3477ae2abad1 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:48.017917 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 357/714] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:48.018605 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2dc41-d909-4277-b0da-962b4891b2a6 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:48.048296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d342594-1428-4e09-bd14-4ac166309fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:48.050426 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d342594-1428-4e09-bd14-4ac166309fa1 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:48.051015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d342594-1428-4e09-bd14-4ac166309fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:48.051454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d342594-1428-4e09-bd14-4ac166309fa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:48.052645 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d342594-1428-4e09-bd14-4ac166309fa1 None admin] RBAC: Adding query filter params (role.id=64c2d69f15ca458c8267c1acfd42d9fc, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=dcd642e290e44e67b56eb3b8361d7af3) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:48.053098 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d342594-1428-4e09-bd14-4ac166309fa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:48.054985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d342594-1428-4e09-bd14-4ac166309fa1 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:48.063709 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 358/715] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:32:47 2022] GET /identity/v3/role_assignments?user.id=dcd642e290e44e67b56eb3b8361d7af3&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=64c2d69f15ca458c8267c1acfd42d9fc => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:48.064794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d342594-1428-4e09-bd14-4ac166309fa1 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:48.123767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:48.189234 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d342594-1428-4e09-bd14-4ac166309fa1 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:48.235641 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2e736-27a5-46d0-8fe5-ca2bd9033bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:48.236827 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2e736-27a5-46d0-8fe5-ca2bd9033bb0 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:48.239054 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2e736-27a5-46d0-8fe5-ca2bd9033bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:48.239574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2e736-27a5-46d0-8fe5-ca2bd9033bb0 None admin] PATH_INFO: `/v3/services/network` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:48.241664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2e736-27a5-46d0-8fe5-ca2bd9033bb0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:48.243519 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2e736-27a5-46d0-8fe5-ca2bd9033bb0 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:48.264497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-ffd2e736-27a5-46d0-8fe5-ca2bd9033bb0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 22 11:32:48.264497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:48.264497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:32:48.264497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:32:48.264497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:32:48.264497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:32:48.264497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:32:48.264497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:32:48.264497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 22 11:32:48.264497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:48.264497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:32:48.264497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:48.264497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:48.264497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:48.264497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:48.264497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:48.264497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:48.265812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:48.265812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:48.265812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:48.265812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:48.265812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:48.265812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:48.265812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:32:48.265812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:32:48.265812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:32:48.265812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:32:48.265812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:48.265812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:48.265812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:48.265812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:48.265812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:48.265812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:48.266634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:48.266634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:48.266634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:48.266634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:48.266634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:48.266634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:48.266634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:48.266634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:48.266634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:48.266634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:48.266634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:32:48.266634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:32:48.266634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 22 11:32:48.266634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:48.269519 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 359/716] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:48 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:48.271089 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2dc41-d909-4277-b0da-962b4891b2a6 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:48.272365 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2e736-27a5-46d0-8fe5-ca2bd9033bb0 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:48.318558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f61bd3-fb41-44a0-9696-ff06813b7c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:48.319640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f61bd3-fb41-44a0-9696-ff06813b7c25 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:48.320185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f61bd3-fb41-44a0-9696-ff06813b7c25 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:48.320660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f61bd3-fb41-44a0-9696-ff06813b7c25 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:48.321559 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f61bd3-fb41-44a0-9696-ff06813b7c25 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:48.322057 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f61bd3-fb41-44a0-9696-ff06813b7c25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:48.324819 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f61bd3-fb41-44a0-9696-ff06813b7c25 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:48.328852 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:48.333030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 358/717] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jun 22 11:32:48 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:48.400038 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2e736-27a5-46d0-8fe5-ca2bd9033bb0 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:48.411777 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:48.442816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ce46a-e762-45c2-a555-5af19488f824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:48.443805 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ce46a-e762-45c2-a555-5af19488f824 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:48.444269 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ce46a-e762-45c2-a555-5af19488f824 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:48.444692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ce46a-e762-45c2-a555-5af19488f824 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:48.445574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ce46a-e762-45c2-a555-5af19488f824 None admin] RBAC: Adding query filter params (name=network) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:48.446046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ce46a-e762-45c2-a555-5af19488f824 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:48.447610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ce46a-e762-45c2-a555-5af19488f824 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:48.453577 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 360/718] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jun 22 11:32:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:48.464516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f61bd3-fb41-44a0-9696-ff06813b7c25 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:48.582751 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f61bd3-fb41-44a0-9696-ff06813b7c25 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:48.625965 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d214b-9421-4be9-8cf0-0db2fe4afd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:48.626708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d214b-9421-4be9-8cf0-0db2fe4afd74 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:48.626996 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d214b-9421-4be9-8cf0-0db2fe4afd74 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:48.627272 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d214b-9421-4be9-8cf0-0db2fe4afd74 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:48.628057 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d214b-9421-4be9-8cf0-0db2fe4afd74 None admin] RBAC: Adding query filter params (type=network) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:48.628365 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d214b-9421-4be9-8cf0-0db2fe4afd74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:48.629743 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d214b-9421-4be9-8cf0-0db2fe4afd74 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:48.636219 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 359/719] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jun 22 11:32:48 2022] GET /identity/v3/services?type=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:49.610694 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e00315-7928-45a2-98ef-40cdf37adfeb None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:49.611221 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e00315-7928-45a2-98ef-40cdf37adfeb None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:49.611600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e00315-7928-45a2-98ef-40cdf37adfeb None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:49.613887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 361/720] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:49.619318 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13251352-9bd6-4c5b-bd7c-2838cece65f6 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:49.619785 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13251352-9bd6-4c5b-bd7c-2838cece65f6 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:49.619881 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13251352-9bd6-4c5b-bd7c-2838cece65f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:49.644875 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-13251352-9bd6-4c5b-bd7c-2838cece65f6 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:49.646078 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-13251352-9bd6-4c5b-bd7c-2838cece65f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:49.662356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93bebb-da5f-4596-b8e0-7e9d1161c79b None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:49.662651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93bebb-da5f-4596-b8e0-7e9d1161c79b None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:49.662908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93bebb-da5f-4596-b8e0-7e9d1161c79b None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:49.664631 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 362/721] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:32:49.669466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4151237-cb1a-4b93-897b-7e0018632ba3 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:49.669752 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4151237-cb1a-4b93-897b-7e0018632ba3 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:49.670031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4151237-cb1a-4b93-897b-7e0018632ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:49.695075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c4151237-cb1a-4b93-897b-7e0018632ba3 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:49.696290 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c4151237-cb1a-4b93-897b-7e0018632ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:49.746922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:49.766095 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 360/722] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:49.781262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 363/723] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 22 11:32:49.793516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13251352-9bd6-4c5b-bd7c-2838cece65f6 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:49.804949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4151237-cb1a-4b93-897b-7e0018632ba3 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:49.930595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13251352-9bd6-4c5b-bd7c-2838cece65f6 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:49.969233 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e03b92-255b-47a6-92d8-8191d2e9ded3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:49.970045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e03b92-255b-47a6-92d8-8191d2e9ded3 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:49.970330 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e03b92-255b-47a6-92d8-8191d2e9ded3 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:49.970607 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e03b92-255b-47a6-92d8-8191d2e9ded3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:49.981709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e03b92-255b-47a6-92d8-8191d2e9ded3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:49.990104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-b6e03b92-255b-47a6-92d8-8191d2e9ded3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:49.990104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:49.990104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:49.990104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:49.990104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:49.990104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:49.990104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:49.990104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:49.990104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:49.990104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:49.990104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:49.990104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:49.990104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:49.990104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:49.990104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:49.990104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:49.991066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:49.991066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:49.991066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:49.991066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:49.991066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:49.991066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:49.991066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:49.991066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:49.991066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:49.991066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:49.991066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:49.991066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:49.991066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:49.991066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:49.991066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:49.991066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:49.991946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:49.991946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:49.991946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:49.991946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:49.991946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:49.991946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:49.991946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:49.991946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:49.991946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:49.991946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:49.991946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:49.991946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:49.991946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 361/724] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:49.997480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4151237-cb1a-4b93-897b-7e0018632ba3 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:49.999870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e03b92-255b-47a6-92d8-8191d2e9ded3 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:50.005536 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:50.083389 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6608d2-7dca-45f7-a107-a68cc8894820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:50.084254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6608d2-7dca-45f7-a107-a68cc8894820 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:50.084542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6608d2-7dca-45f7-a107-a68cc8894820 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:50.084817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6608d2-7dca-45f7-a107-a68cc8894820 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:50.106847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6608d2-7dca-45f7-a107-a68cc8894820 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:50.112025 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:50.123793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-2a6608d2-7dca-45f7-a107-a68cc8894820 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:32:50.123793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:50.123793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:50.123793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:50.123793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:50.123793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:50.123793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:50.123793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:50.123793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:50.123793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:50.123793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:50.123793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:50.123793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:50.123793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:50.123793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:50.123793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:50.124756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:50.124756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:50.124756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:50.124756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:50.124756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:50.124756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:50.124756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:50.124756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:50.124756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:50.124756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:50.124756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:50.124756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:50.124756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:50.124756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:50.124756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:50.124756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:50.125666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:50.125666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:50.125666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:50.125666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:50.125666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:50.125666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:50.125666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:50.125666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:50.125666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:50.125666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:50.125666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:32:50.125666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:50.127665 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 364/725] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:32:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:32:50.131923 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6608d2-7dca-45f7-a107-a68cc8894820 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:50.185735 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e03b92-255b-47a6-92d8-8191d2e9ded3 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:50.226960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c47f59-eb32-4988-86a6-2ae9d67e5778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:50.229820 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c47f59-eb32-4988-86a6-2ae9d67e5778 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:50.230332 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c47f59-eb32-4988-86a6-2ae9d67e5778 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:50.230621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c47f59-eb32-4988-86a6-2ae9d67e5778 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:50.231501 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c47f59-eb32-4988-86a6-2ae9d67e5778 None admin] RBAC: Adding query filter params (name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:50.231875 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c47f59-eb32-4988-86a6-2ae9d67e5778 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:50.234554 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c47f59-eb32-4988-86a6-2ae9d67e5778 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:50.259641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 362/726] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:32:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:50.264305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c47f59-eb32-4988-86a6-2ae9d67e5778 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:50.334525 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6608d2-7dca-45f7-a107-a68cc8894820 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:50.395755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ea4d5-0444-49c7-859f-f1ae712da417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:50.396440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ea4d5-0444-49c7-859f-f1ae712da417 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:50.396720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ea4d5-0444-49c7-859f-f1ae712da417 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:50.396991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ea4d5-0444-49c7-859f-f1ae712da417 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:50.397753 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ea4d5-0444-49c7-859f-f1ae712da417 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:50.398057 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ea4d5-0444-49c7-859f-f1ae712da417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:50.399482 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ea4d5-0444-49c7-859f-f1ae712da417 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:50.417831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 365/727] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:50.419043 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ea4d5-0444-49c7-859f-f1ae712da417 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:50.468589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:50.470103 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c47f59-eb32-4988-86a6-2ae9d67e5778 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:50.499015 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:50.534825 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fe2e1-1cfb-4e6c-96e2-2e62c53a5afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:50.535660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fe2e1-1cfb-4e6c-96e2-2e62c53a5afc None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:50.535946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fe2e1-1cfb-4e6c-96e2-2e62c53a5afc None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:50.536222 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fe2e1-1cfb-4e6c-96e2-2e62c53a5afc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:50.555982 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710fe2e1-1cfb-4e6c-96e2-2e62c53a5afc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:50.557462 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710fe2e1-1cfb-4e6c-96e2-2e62c53a5afc None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:50.571392 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ea4d5-0444-49c7-859f-f1ae712da417 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:50.591411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-710fe2e1-1cfb-4e6c-96e2-2e62c53a5afc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:50.591411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:50.591411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:50.591411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:50.591411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:50.591411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:50.591411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:50.591411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:50.591411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:50.591411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:50.591411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:50.591411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:50.591411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:50.591411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:50.591411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:50.592358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:50.592358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:50.592358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:50.592358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:50.592358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:50.592358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:50.592358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:50.592358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:50.592358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:50.592358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:50.592358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:50.592358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:50.592358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:50.592358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:50.592358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:50.592358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:50.592358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:50.593310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:50.593310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:50.593310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:50.593310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:50.602341 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 363/728] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:50.603918 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fe2e1-1cfb-4e6c-96e2-2e62c53a5afc None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:50.605960 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b30330-f4ea-41fe-aac6-e830ee28443a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:50.607008 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b30330-f4ea-41fe-aac6-e830ee28443a None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:50.607432 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b30330-f4ea-41fe-aac6-e830ee28443a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:50.607806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b30330-f4ea-41fe-aac6-e830ee28443a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:50.619496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b30330-f4ea-41fe-aac6-e830ee28443a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:50.621062 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b30330-f4ea-41fe-aac6-e830ee28443a None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:50.639521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-d6b30330-f4ea-41fe-aac6-e830ee28443a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:50.639521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:50.639521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:50.639521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:50.639521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:50.639521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:50.639521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:50.639521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:50.639521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:50.639521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:50.639521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:50.639521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:50.639521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:50.639521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:50.639521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:50.640760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:50.640760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:50.640760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:50.640760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:50.640760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:50.640760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:50.640760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:50.640760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:50.640760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:50.640760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:50.640760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:50.640760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:50.640760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:50.640760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:50.640760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:50.640760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:50.640760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:50.641684 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:50.641684 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:50.641684 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:50.641684 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:50.641684 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 366/729] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:50.647374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b30330-f4ea-41fe-aac6-e830ee28443a None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:50.752212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:50.778446 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b30330-f4ea-41fe-aac6-e830ee28443a None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:50.815292 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8a33f-9511-43ad-9f79-ca529c8c6397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:50.816011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fe2e1-1cfb-4e6c-96e2-2e62c53a5afc None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:50.816542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8a33f-9511-43ad-9f79-ca529c8c6397 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:50.816542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8a33f-9511-43ad-9f79-ca529c8c6397 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:50.816859 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8a33f-9511-43ad-9f79-ca529c8c6397 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:50.817564 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8a33f-9511-43ad-9f79-ca529c8c6397 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:50.817869 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8a33f-9511-43ad-9f79-ca529c8c6397 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:50.819402 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8a33f-9511-43ad-9f79-ca529c8c6397 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:50.832481 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 367/730] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:50.838785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8a33f-9511-43ad-9f79-ca529c8c6397 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:50.870368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f5c96-53da-461c-afe7-f2184f5e682e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:50.871317 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f5c96-53da-461c-afe7-f2184f5e682e None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:50.871634 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f5c96-53da-461c-afe7-f2184f5e682e None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:50.871916 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f5c96-53da-461c-afe7-f2184f5e682e None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:50.872992 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f5c96-53da-461c-afe7-f2184f5e682e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:50.873708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f5c96-53da-461c-afe7-f2184f5e682e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:50.875455 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f5c96-53da-461c-afe7-f2184f5e682e None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:50.894006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 364/731] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:50.895462 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a308331-c93a-4ec9-9089-c26ebe35ff4b None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:50.895753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a308331-c93a-4ec9-9089-c26ebe35ff4b None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:50.896028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a308331-c93a-4ec9-9089-c26ebe35ff4b None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:50.897599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 365/732] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:50.899441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a308331-c93a-4ec9-9089-c26ebe35ff4b None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:50.938512 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:50.959386 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8a33f-9511-43ad-9f79-ca529c8c6397 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:50.996950 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e3fd1-bc26-4c43-9263-84c2e583453f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:50.997892 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e3fd1-bc26-4c43-9263-84c2e583453f None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:50.998283 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e3fd1-bc26-4c43-9263-84c2e583453f None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:50.998768 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e3fd1-bc26-4c43-9263-84c2e583453f None admin] PATH_INFO: `/v3/users/nova` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:51.002806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:51.010205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e3fd1-bc26-4c43-9263-84c2e583453f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:51.011811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e3fd1-bc26-4c43-9263-84c2e583453f None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:51.027467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-4f4e3fd1-bc26-4c43-9263-84c2e583453f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 22 11:32:51.027467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:51.027467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:51.027467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:51.027467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:51.027467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:51.027467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:51.027467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:51.027467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:51.027467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:51.027467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:51.027467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:51.027467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:51.027467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:51.027467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:51.027467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:51.028460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:51.028460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:51.028460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:51.028460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:51.028460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:51.028460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:51.028460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:51.028460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:51.028460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:51.028460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:51.028460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:51.028460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:51.028460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:51.028460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:51.028460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:51.028460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:51.028460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:51.028460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:51.029416 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:51.029416 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:51.029416 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:51.029416 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:51.029416 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:51.029416 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:51.029416 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:51.029416 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:51.029416 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 22 11:32:51.029416 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:51.033325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 368/733] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jun 22 11:32:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:32:51.034759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac2492-5423-41ef-aec4-a289e28694e2 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:51.035164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac2492-5423-41ef-aec4-a289e28694e2 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:51.035386 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac2492-5423-41ef-aec4-a289e28694e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:51.044422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a308331-c93a-4ec9-9089-c26ebe35ff4b None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:51.054891 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-3bac2492-5423-41ef-aec4-a289e28694e2 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:51.057850 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-3bac2492-5423-41ef-aec4-a289e28694e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:51.083886 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d08ce-8ba8-49a8-835c-832fb07b9ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:51.084696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d08ce-8ba8-49a8-835c-832fb07b9ad9 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:51.084978 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d08ce-8ba8-49a8-835c-832fb07b9ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:51.085302 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d08ce-8ba8-49a8-835c-832fb07b9ad9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:51.092803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d08ce-8ba8-49a8-835c-832fb07b9ad9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:51.094242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d08ce-8ba8-49a8-835c-832fb07b9ad9 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:51.109907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-c38d08ce-8ba8-49a8-835c-832fb07b9ad9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 22 11:32:51.109907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:51.109907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:51.109907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:51.109907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:51.109907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:51.109907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:51.109907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:51.109907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:51.109907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:51.109907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:51.109907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:51.109907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:51.109907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:51.109907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:51.109907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:51.111003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:51.111003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:51.111003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:51.111003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:51.111003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:51.111003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:51.111003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:51.111003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:51.111003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:51.111003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:51.111003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:51.111003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:51.111003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:51.111003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:51.111003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:51.111003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:51.111003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:51.111003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:51.111989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:51.111989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:51.111989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:51.111989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:51.111989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:51.111989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:51.111989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:51.111989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:51.111989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 22 11:32:51.111989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:51.111989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 366/734] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jun 22 11:32:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:51.113095 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d08ce-8ba8-49a8-835c-832fb07b9ad9 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:51.138431 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 369/735] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:51.140479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3464d-53a3-4c35-95e3-7e511124a59f None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:51.141007 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3464d-53a3-4c35-95e3-7e511124a59f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:51.141489 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3464d-53a3-4c35-95e3-7e511124a59f None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:51.146910 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 370/736] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:51.147805 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3464d-53a3-4c35-95e3-7e511124a59f None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:51.226804 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:51.241912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d08ce-8ba8-49a8-835c-832fb07b9ad9 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:51.267118 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3464d-53a3-4c35-95e3-7e511124a59f None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:51.278705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa8d11-a0eb-497a-8db9-66e02274b46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:51.279414 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa8d11-a0eb-497a-8db9-66e02274b46b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:51.279688 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa8d11-a0eb-497a-8db9-66e02274b46b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:51.279965 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa8d11-a0eb-497a-8db9-66e02274b46b None admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:51.280813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa8d11-a0eb-497a-8db9-66e02274b46b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:51.281126 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa8d11-a0eb-497a-8db9-66e02274b46b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:51.282588 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa8d11-a0eb-497a-8db9-66e02274b46b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:51.295704 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:51.302515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 367/737] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jun 22 11:32:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:51.304123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76aba81-e0b9-4097-858f-7acceea5ebb7 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:51.304418 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76aba81-e0b9-4097-858f-7acceea5ebb7 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:51.304723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76aba81-e0b9-4097-858f-7acceea5ebb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:51.306162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e445b7a-bd14-4d1a-85b9-fe3896498e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:51.306938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e445b7a-bd14-4d1a-85b9-fe3896498e54 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:51.307113 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e445b7a-bd14-4d1a-85b9-fe3896498e54 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:51.307422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e445b7a-bd14-4d1a-85b9-fe3896498e54 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:51.308266 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e445b7a-bd14-4d1a-85b9-fe3896498e54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:51.308574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e445b7a-bd14-4d1a-85b9-fe3896498e54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:51.310017 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e445b7a-bd14-4d1a-85b9-fe3896498e54 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:51.320942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d76aba81-e0b9-4097-858f-7acceea5ebb7 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:51.322152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d76aba81-e0b9-4097-858f-7acceea5ebb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:51.328262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 371/738] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jun 22 11:32:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:51.329079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e445b7a-bd14-4d1a-85b9-fe3896498e54 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:51.392350 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 368/739] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:51.393776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e591b74-a2b0-4753-8905-2b3ebfccc0c8 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:51.394063 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e591b74-a2b0-4753-8905-2b3ebfccc0c8 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:51.394337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e591b74-a2b0-4753-8905-2b3ebfccc0c8 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:51.395616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 369/740] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:51.399645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e591b74-a2b0-4753-8905-2b3ebfccc0c8 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:51.432288 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e445b7a-bd14-4d1a-85b9-fe3896498e54 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:51.466934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60f8b0-acd4-4671-90d8-fca8aa6ce4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:51.467771 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60f8b0-acd4-4671-90d8-fca8aa6ce4ef None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:51.468053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60f8b0-acd4-4671-90d8-fca8aa6ce4ef None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:51.468330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60f8b0-acd4-4671-90d8-fca8aa6ce4ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:51.480593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60f8b0-acd4-4671-90d8-fca8aa6ce4ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:51.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-bc60f8b0-acd4-4671-90d8-fca8aa6ce4ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:32:51.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:51.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:51.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:51.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:51.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:51.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:51.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:51.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:51.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:51.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:51.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:51.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:51.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:51.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:51.488756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:51.489760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:51.489760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:51.489760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:51.489760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:51.489760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:51.489760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:51.489760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:51.489760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:51.489760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:51.489760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:51.489760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:51.489760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:51.489760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:51.489760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:51.489760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:51.489760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:51.491031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:51.491031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:51.491031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:51.491031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:51.491031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:51.491031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:51.491031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:51.491031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:51.491031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:51.491031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:51.491031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:32:51.491031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:51.491966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:51.492055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 372/741] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:32:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:51.492055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60f8b0-acd4-4671-90d8-fca8aa6ce4ef None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:51.524383 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e591b74-a2b0-4753-8905-2b3ebfccc0c8 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:51.539527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:51.562286 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8b040-2005-438b-93a7-d3c6f1ec43ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:51.563091 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8b040-2005-438b-93a7-d3c6f1ec43ca None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:51.563391 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8b040-2005-438b-93a7-d3c6f1ec43ca None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:51.563674 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8b040-2005-438b-93a7-d3c6f1ec43ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:51.574646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8b040-2005-438b-93a7-d3c6f1ec43ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:51.576169 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8b040-2005-438b-93a7-d3c6f1ec43ca None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:51.594136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-d7b8b040-2005-438b-93a7-d3c6f1ec43ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:51.594136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:51.594136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:51.594136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:51.594136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:51.594136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:51.594136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:51.594136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:51.594136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:51.594136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:51.594136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:51.594136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:51.594136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:51.594136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:51.594136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:51.595179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:51.595179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:51.595179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:51.595179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:51.595179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:51.595179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:51.595179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:51.595179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:51.595179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:51.595179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:51.595179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:51.595179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:51.595179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:51.595179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:51.595179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:51.595179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:51.595179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:51.596122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:51.596122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:51.596122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:51.596122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:51.596122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 370/742] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:51.597520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6363e7e-46e2-4c93-bbf8-5692634b55f2 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:51.597809 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6363e7e-46e2-4c93-bbf8-5692634b55f2 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:51.598090 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6363e7e-46e2-4c93-bbf8-5692634b55f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:51.606848 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60f8b0-acd4-4671-90d8-fca8aa6ce4ef None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:51.618681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-e6363e7e-46e2-4c93-bbf8-5692634b55f2 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:51.620829 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e6363e7e-46e2-4c93-bbf8-5692634b55f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:51.641093 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad671c-2c32-4414-9276-05202581cbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:51.641744 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad671c-2c32-4414-9276-05202581cbc5 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:51.642029 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad671c-2c32-4414-9276-05202581cbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:51.642302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad671c-2c32-4414-9276-05202581cbc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:51.694087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 371/743] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 22 11:32:51.694900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6363e7e-46e2-4c93-bbf8-5692634b55f2 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:51.711023 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad671c-2c32-4414-9276-05202581cbc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:51.712625 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad671c-2c32-4414-9276-05202581cbc5 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:51.757215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:51.765807 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:51.806325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6363e7e-46e2-4c93-bbf8-5692634b55f2 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:51.811908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 373/744] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 22 11:32:51.812751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad671c-2c32-4414-9276-05202581cbc5 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:51.839903 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd907052-3b75-4198-9f0d-9bcb3c2f2402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:51.840574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd907052-3b75-4198-9f0d-9bcb3c2f2402 None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:51.840848 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd907052-3b75-4198-9f0d-9bcb3c2f2402 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:51.841120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd907052-3b75-4198-9f0d-9bcb3c2f2402 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:51.841829 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd907052-3b75-4198-9f0d-9bcb3c2f2402 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:51.843202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd907052-3b75-4198-9f0d-9bcb3c2f2402 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:51.857309 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 372/745] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jun 22 11:32:51 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:32:51.858410 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd907052-3b75-4198-9f0d-9bcb3c2f2402 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:51.945766 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad671c-2c32-4414-9276-05202581cbc5 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:51.984973 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd907052-3b75-4198-9f0d-9bcb3c2f2402 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:51.995705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff72918-d52d-4067-b024-58d8fd54001b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:51.996387 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff72918-d52d-4067-b024-58d8fd54001b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:51.996671 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff72918-d52d-4067-b024-58d8fd54001b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:51.996943 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff72918-d52d-4067-b024-58d8fd54001b None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:51.997692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff72918-d52d-4067-b024-58d8fd54001b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:51.998021 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff72918-d52d-4067-b024-58d8fd54001b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:51.999402 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff72918-d52d-4067-b024-58d8fd54001b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:52.001458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:52.016333 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 374/746] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:52.016333 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff72918-d52d-4067-b024-58d8fd54001b None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:52.019040 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:52.033214 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086bb75-38ad-4f24-bfa6-6915e0fd2282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:52.033882 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086bb75-38ad-4f24-bfa6-6915e0fd2282 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:52.034746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086bb75-38ad-4f24-bfa6-6915e0fd2282 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:52.035027 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086bb75-38ad-4f24-bfa6-6915e0fd2282 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:52.035787 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086bb75-38ad-4f24-bfa6-6915e0fd2282 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:52.036102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086bb75-38ad-4f24-bfa6-6915e0fd2282 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:52.037494 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086bb75-38ad-4f24-bfa6-6915e0fd2282 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:52.050021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 373/747] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:52.050570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086bb75-38ad-4f24-bfa6-6915e0fd2282 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:52.128831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff72918-d52d-4067-b024-58d8fd54001b None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:52.158673 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086bb75-38ad-4f24-bfa6-6915e0fd2282 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:52.164864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb12f3f-622d-447f-931d-4f1cc60b234d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:52.165532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb12f3f-622d-447f-931d-4f1cc60b234d None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:52.165815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb12f3f-622d-447f-931d-4f1cc60b234d None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:52.166085 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb12f3f-622d-447f-931d-4f1cc60b234d None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:52.166825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb12f3f-622d-447f-931d-4f1cc60b234d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:52.168330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb12f3f-622d-447f-931d-4f1cc60b234d None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:52.191379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 375/748] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jun 22 11:32:52 2022] POST /identity/v3/services => generated 240 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:32:52.191379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb12f3f-622d-447f-931d-4f1cc60b234d None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:52.214874 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9625a57-eb0b-4016-bfbf-cfdad80a7aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:52.215849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9625a57-eb0b-4016-bfbf-cfdad80a7aad None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:52.216120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9625a57-eb0b-4016-bfbf-cfdad80a7aad None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:52.216398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9625a57-eb0b-4016-bfbf-cfdad80a7aad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:52.230107 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9625a57-eb0b-4016-bfbf-cfdad80a7aad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:52.231852 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9625a57-eb0b-4016-bfbf-cfdad80a7aad None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:52.253196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-e9625a57-eb0b-4016-bfbf-cfdad80a7aad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:52.253196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:52.253196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:52.253196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:52.253196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:52.253196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:52.253196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:52.253196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:52.253196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:52.253196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:52.253196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:52.253196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:52.253196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:52.253196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:52.253196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:52.254172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:52.254172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:52.254172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:52.254172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:52.254172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:52.254172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:52.254172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:52.254172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:52.254172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:52.254172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:52.254172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:52.254172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:52.254172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:52.254172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:52.254172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:52.254172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:52.254172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:52.255119 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:52.255119 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:52.255119 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:52.255119 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:52.255119 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 374/749] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:52.255902 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9625a57-eb0b-4016-bfbf-cfdad80a7aad None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:52.286491 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:52.381159 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9625a57-eb0b-4016-bfbf-cfdad80a7aad None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:52.397144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:52.428976 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0835a84-dfa7-4dcc-95e0-cc4f91374a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:52.429899 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0835a84-dfa7-4dcc-95e0-cc4f91374a7b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:52.430307 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0835a84-dfa7-4dcc-95e0-cc4f91374a7b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:52.430590 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0835a84-dfa7-4dcc-95e0-cc4f91374a7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:52.439178 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0835a84-dfa7-4dcc-95e0-cc4f91374a7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:52.441021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0835a84-dfa7-4dcc-95e0-cc4f91374a7b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:52.452239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb12f3f-622d-447f-931d-4f1cc60b234d None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:52.455867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-d0835a84-dfa7-4dcc-95e0-cc4f91374a7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:52.455867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:52.455867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:52.455867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:52.455867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:52.455867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:52.455867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:52.455867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:52.455867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:52.455867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:52.455867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:52.455867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:52.455867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:52.455867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:52.455867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:52.457292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:52.457292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:52.457292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:52.457292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:52.457292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:52.457292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:52.457292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:52.457292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:52.457292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:52.457292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:52.457292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:52.457292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:52.457292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:52.457292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:52.457292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:52.457292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:52.457292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:52.458190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:52.458190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:52.458190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:52.458190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:52.458190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:52.458190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:52.458190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:52.458190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:52.458190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 375/750] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:52.458852 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0835a84-dfa7-4dcc-95e0-cc4f91374a7b None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:52.491514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517b068-2d03-4999-9751-60816c16c21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:52.492469 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517b068-2d03-4999-9751-60816c16c21f None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:52.492856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517b068-2d03-4999-9751-60816c16c21f None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:52.493251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517b068-2d03-4999-9751-60816c16c21f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:52.504816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517b068-2d03-4999-9751-60816c16c21f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:52.506468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517b068-2d03-4999-9751-60816c16c21f None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:52.528315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-9517b068-2d03-4999-9751-60816c16c21f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:52.528315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:52.528315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:52.528315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:52.528315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:52.528315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:52.528315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:52.528315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:52.528315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:52.528315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:52.528315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:52.528315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:52.528315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:52.528315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:52.528315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:52.529474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:52.529474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:52.529474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:52.529474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:52.529474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:52.529474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:52.529474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:52.529474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:52.529474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:52.529474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:52.529474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:52.529474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:52.529474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:52.529474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:52.529474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:52.529474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:52.529474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:52.530359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:52.530359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:52.530359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:52.530359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:52.532176 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 376/751] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:52.532176 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517b068-2d03-4999-9751-60816c16c21f None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:52.566007 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:52.592556 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0835a84-dfa7-4dcc-95e0-cc4f91374a7b None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:52.647104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb362b-371d-4038-9cfc-6fe842344cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:52.648003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb362b-371d-4038-9cfc-6fe842344cae None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:52.648289 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb362b-371d-4038-9cfc-6fe842344cae None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:52.648568 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb362b-371d-4038-9cfc-6fe842344cae None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:52.650645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb362b-371d-4038-9cfc-6fe842344cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:52.651043 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb362b-371d-4038-9cfc-6fe842344cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:52.652411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb362b-371d-4038-9cfc-6fe842344cae None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:52.658027 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517b068-2d03-4999-9751-60816c16c21f None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:52.678101 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 376/752] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:52.678101 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb362b-371d-4038-9cfc-6fe842344cae None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:52.679725 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:52.701980 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257c648-5e0c-4f38-859a-21661f6418b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:52.702706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257c648-5e0c-4f38-859a-21661f6418b4 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:52.702993 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257c648-5e0c-4f38-859a-21661f6418b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:52.703272 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257c648-5e0c-4f38-859a-21661f6418b4 None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:52.704348 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257c648-5e0c-4f38-859a-21661f6418b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:52.704477 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257c648-5e0c-4f38-859a-21661f6418b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:52.706573 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257c648-5e0c-4f38-859a-21661f6418b4 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:52.719533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 377/753] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:52.720902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257c648-5e0c-4f38-859a-21661f6418b4 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:52.818456 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb362b-371d-4038-9cfc-6fe842344cae None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:52.842624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257c648-5e0c-4f38-859a-21661f6418b4 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:52.856788 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82e5b8-39eb-429f-9ce4-5d6d6596e58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:52.857461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82e5b8-39eb-429f-9ce4-5d6d6596e58d None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:52.857751 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82e5b8-39eb-429f-9ce4-5d6d6596e58d None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:52.858032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82e5b8-39eb-429f-9ce4-5d6d6596e58d None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:52.858983 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82e5b8-39eb-429f-9ce4-5d6d6596e58d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:52.859093 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82e5b8-39eb-429f-9ce4-5d6d6596e58d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:52.860741 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e82e5b8-39eb-429f-9ce4-5d6d6596e58d None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:52.877591 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 377/754] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:52.877591 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82e5b8-39eb-429f-9ce4-5d6d6596e58d None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:52.881143 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:52.902451 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a0a19-d313-451d-9bc8-a7649db20d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:52.904875 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a0a19-d313-451d-9bc8-a7649db20d9c None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:52.905524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a0a19-d313-451d-9bc8-a7649db20d9c None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:52.907255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a0a19-d313-451d-9bc8-a7649db20d9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:52.916699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a0a19-d313-451d-9bc8-a7649db20d9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:52.918314 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a0a19-d313-451d-9bc8-a7649db20d9c None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:52.933590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-949a0a19-d313-451d-9bc8-a7649db20d9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:52.933590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:52.933590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:52.933590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:52.933590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:52.933590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:52.933590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:52.933590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:52.933590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:52.933590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:52.933590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:52.933590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:52.933590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:52.933590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:52.933590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:52.934604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:52.934604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:52.934604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:52.934604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:52.934604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:52.934604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:52.934604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:52.934604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:52.934604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:52.934604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:52.934604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:52.934604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:52.934604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:52.934604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:52.934604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:52.934604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:52.934604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:52.935522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:52.935522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:52.935522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:52.935522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:52.935522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:52.935522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:52.935522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:52.935522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:52.939810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 378/755] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:52.940912 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a0a19-d313-451d-9bc8-a7649db20d9c None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:52.987193 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:53.009258 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e82e5b8-39eb-429f-9ce4-5d6d6596e58d None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:53.059727 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66610b2a-e6c3-443a-8878-f0983fb46666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:53.061125 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66610b2a-e6c3-443a-8878-f0983fb46666 None admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:53.061478 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66610b2a-e6c3-443a-8878-f0983fb46666 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:53.061970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66610b2a-e6c3-443a-8878-f0983fb46666 None admin] PATH_INFO: `/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/479a376dcac14c509c1c55f0046d88cf/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:53.064147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66610b2a-e6c3-443a-8878-f0983fb46666 None admin] RBAC: Authorizing `identity:create_grant(project_id=47d2bb094c9843c9a7726f1d672d2433, user_id=479a376dcac14c509c1c55f0046d88cf, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:53.064264 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a0a19-d313-451d-9bc8-a7649db20d9c None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:53.066166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66610b2a-e6c3-443a-8878-f0983fb46666 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:53.085744 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 378/756] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:32:52 2022] PUT /identity/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/479a376dcac14c509c1c55f0046d88cf/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:32:53.086644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66610b2a-e6c3-443a-8878-f0983fb46666 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:53.122083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195e421-2674-4498-876f-a37b3fdd7b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:53.124074 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195e421-2674-4498-876f-a37b3fdd7b64 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:53.124951 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195e421-2674-4498-876f-a37b3fdd7b64 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:53.125425 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195e421-2674-4498-876f-a37b3fdd7b64 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:53.137426 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195e421-2674-4498-876f-a37b3fdd7b64 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:53.140788 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195e421-2674-4498-876f-a37b3fdd7b64 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:53.160073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-9195e421-2674-4498-876f-a37b3fdd7b64 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 22 11:32:53.160073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:53.160073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:53.160073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:53.160073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:53.160073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:53.160073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:53.160073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:53.160073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:53.160073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:53.160073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:53.160073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:53.160073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:53.160073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:53.160073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:53.160073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:53.161075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:53.161075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:53.161075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:53.161075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:53.161075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:53.161075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:53.161075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:53.161075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:53.161075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:53.161075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:53.161075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:53.161075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:53.161075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:53.161075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:53.161075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:53.161075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:53.161075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:53.161075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:53.161999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:53.161999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:53.161999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:53.161999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:53.161999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:53.161999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:53.161999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:53.161999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:53.161999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 22 11:32:53.161999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:53.164938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 379/757] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:32:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:53.172428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195e421-2674-4498-876f-a37b3fdd7b64 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:53.200995 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:53.288570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66610b2a-e6c3-443a-8878-f0983fb46666 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:53.330408 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599761a-14a7-4883-b60f-4a9342aa0486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:53.331901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599761a-14a7-4883-b60f-4a9342aa0486 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:53.332330 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599761a-14a7-4883-b60f-4a9342aa0486 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:53.332779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599761a-14a7-4883-b60f-4a9342aa0486 None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:53.333720 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599761a-14a7-4883-b60f-4a9342aa0486 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:53.334173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599761a-14a7-4883-b60f-4a9342aa0486 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:53.335930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599761a-14a7-4883-b60f-4a9342aa0486 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:53.350502 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 379/758] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:53.356594 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599761a-14a7-4883-b60f-4a9342aa0486 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:53.358286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:53.369569 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195e421-2674-4498-876f-a37b3fdd7b64 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:53.410444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab418f-bdd6-46d0-85e6-c472a8753637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:53.411444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab418f-bdd6-46d0-85e6-c472a8753637 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:53.411832 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab418f-bdd6-46d0-85e6-c472a8753637 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:53.412207 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab418f-bdd6-46d0-85e6-c472a8753637 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:53.413176 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab418f-bdd6-46d0-85e6-c472a8753637 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:53.413608 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab418f-bdd6-46d0-85e6-c472a8753637 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:53.415137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab418f-bdd6-46d0-85e6-c472a8753637 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:53.434531 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 380/759] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jun 22 11:32:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:53.440216 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab418f-bdd6-46d0-85e6-c472a8753637 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:53.474451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599761a-14a7-4883-b60f-4a9342aa0486 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:53.502065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:53.554712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea87ca9-c3f4-4e69-abc4-8ebc810736d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:53.555513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea87ca9-c3f4-4e69-abc4-8ebc810736d2 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:53.555799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea87ca9-c3f4-4e69-abc4-8ebc810736d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:53.556077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea87ca9-c3f4-4e69-abc4-8ebc810736d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:53.556975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea87ca9-c3f4-4e69-abc4-8ebc810736d2 None admin] RBAC: Adding query filter params (role.id=64c2d69f15ca458c8267c1acfd42d9fc, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=955c44ffe1bd426e900c7405df44d1cf) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:53.557224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea87ca9-c3f4-4e69-abc4-8ebc810736d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:53.558782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea87ca9-c3f4-4e69-abc4-8ebc810736d2 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:53.568743 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 380/760] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:32:53 2022] GET /identity/v3/role_assignments?user.id=955c44ffe1bd426e900c7405df44d1cf&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=64c2d69f15ca458c8267c1acfd42d9fc => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:53.575875 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab418f-bdd6-46d0-85e6-c472a8753637 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:53.615005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d2852-8a84-4542-90bd-63e257d65ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:53.615954 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d2852-8a84-4542-90bd-63e257d65ff0 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:53.616416 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d2852-8a84-4542-90bd-63e257d65ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:53.616841 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d2852-8a84-4542-90bd-63e257d65ff0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:53.641284 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:53.692332 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d2852-8a84-4542-90bd-63e257d65ff0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:53.693703 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d2852-8a84-4542-90bd-63e257d65ff0 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:53.801402 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 381/761] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 22 11:32:53.806799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea87ca9-c3f4-4e69-abc4-8ebc810736d2 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:53.957779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea87ca9-c3f4-4e69-abc4-8ebc810736d2 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:54.022240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094ae90-d49b-4918-97cb-524c046992fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:54.023127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094ae90-d49b-4918-97cb-524c046992fb None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:54.023438 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094ae90-d49b-4918-97cb-524c046992fb None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:54.023723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094ae90-d49b-4918-97cb-524c046992fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:54.045475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094ae90-d49b-4918-97cb-524c046992fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:54.046979 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094ae90-d49b-4918-97cb-524c046992fb None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:54.068106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-3094ae90-d49b-4918-97cb-524c046992fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:54.068106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:54.068106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:54.068106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:54.068106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:54.068106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:54.068106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:54.068106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:54.068106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:54.068106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:54.068106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:54.068106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:54.068106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:54.068106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:54.068106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:54.069076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:54.069076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:54.069076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:54.069076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:54.069076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:54.069076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:54.069076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:54.069076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:54.069076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:54.069076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:54.069076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:54.069076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:54.069076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:54.069076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:54.069076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:54.069076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:54.069076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:54.069981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:54.069981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:54.069981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:54.069981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:54.069981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 381/762] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:54.075175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d2852-8a84-4542-90bd-63e257d65ff0 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:54.148225 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f872cf-e331-4105-9bb7-464561ffc1fe None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:54.148939 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f872cf-e331-4105-9bb7-464561ffc1fe None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:54.149503 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f872cf-e331-4105-9bb7-464561ffc1fe None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:54.155273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:54.155538 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 382/763] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:54.157324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bb40f-d91e-4a1a-ac35-4b3dbd5451e5 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:54.157842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bb40f-d91e-4a1a-ac35-4b3dbd5451e5 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:54.158356 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640bb40f-d91e-4a1a-ac35-4b3dbd5451e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:54.196490 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-640bb40f-d91e-4a1a-ac35-4b3dbd5451e5 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:54.197979 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-640bb40f-d91e-4a1a-ac35-4b3dbd5451e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:54.203375 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d2852-8a84-4542-90bd-63e257d65ff0 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:54.238353 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810a421-a2cc-4951-a093-124f0bef44cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:54.239022 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810a421-a2cc-4951-a093-124f0bef44cd None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:54.239302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810a421-a2cc-4951-a093-124f0bef44cd None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:54.239604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810a421-a2cc-4951-a093-124f0bef44cd None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:54.240448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810a421-a2cc-4951-a093-124f0bef44cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:54.240760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810a421-a2cc-4951-a093-124f0bef44cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:54.242117 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810a421-a2cc-4951-a093-124f0bef44cd None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:54.254788 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 382/764] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:54.263289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810a421-a2cc-4951-a093-124f0bef44cd None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:54.275087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:54.375683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810a421-a2cc-4951-a093-124f0bef44cd None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:54.385136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 383/765] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:32:54.411400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fd720-bdc7-4d15-a453-9d50d62300e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:54.412527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fd720-bdc7-4d15-a453-9d50d62300e8 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:54.412990 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fd720-bdc7-4d15-a453-9d50d62300e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:54.413443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fd720-bdc7-4d15-a453-9d50d62300e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:54.415537 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:54.421891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640bb40f-d91e-4a1a-ac35-4b3dbd5451e5 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:54.422277 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36fd720-bdc7-4d15-a453-9d50d62300e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:54.425916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36fd720-bdc7-4d15-a453-9d50d62300e8 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:54.454437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-b36fd720-bdc7-4d15-a453-9d50d62300e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:54.454437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:54.454437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:54.454437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:54.454437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:54.454437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:54.454437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:54.454437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:54.454437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:54.454437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:54.454437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:54.454437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:54.454437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:54.454437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:54.454437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:54.455556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:54.455556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:54.455556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:54.455556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:54.455556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:54.455556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:54.455556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:54.455556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:54.455556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:54.455556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:54.455556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:54.455556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:54.455556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:54.455556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:54.455556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:54.455556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:54.455556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:54.457251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:54.457251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:54.457251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:54.457251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:54.457251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:54.457251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:54.457251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:54.457251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:54.457251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 383/766] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:54.461207 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fd720-bdc7-4d15-a453-9d50d62300e8 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:54.552359 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640bb40f-d91e-4a1a-ac35-4b3dbd5451e5 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:54.592677 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730314f-36a4-46a0-bebc-ad6d04c322f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:54.593673 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730314f-36a4-46a0-bebc-ad6d04c322f3 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:54.594058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730314f-36a4-46a0-bebc-ad6d04c322f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:54.594437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730314f-36a4-46a0-bebc-ad6d04c322f3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:54.595200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730314f-36a4-46a0-bebc-ad6d04c322f3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:54.597412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730314f-36a4-46a0-bebc-ad6d04c322f3 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:54.611589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-b730314f-36a4-46a0-bebc-ad6d04c322f3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 22 11:32:54.611589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:54.611589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:32:54.611589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:32:54.611589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:32:54.611589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:32:54.611589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:32:54.611589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:32:54.611589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 22 11:32:54.611589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:54.611589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:32:54.611589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:54.611589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:54.611589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:54.611589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:54.611589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:54.613167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:54.613167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:54.613167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:54.613167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:54.613167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:54.613167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:54.613167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:54.613167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:32:54.613167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:32:54.613167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:32:54.613167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:32:54.613167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:54.613167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:54.613167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:54.613167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:54.613167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:54.613984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:54.613984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:54.613984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:54.613984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:54.613984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:54.613984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:54.613984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:54.613984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:54.613984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:54.613984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:54.613984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:54.613984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:32:54.613984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:32:54.613984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 22 11:32:54.613984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:54.615545 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 384/767] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Jun 22 11:32:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:32:54.620543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730314f-36a4-46a0-bebc-ad6d04c322f3 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:54.657769 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:54.714145 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fd720-bdc7-4d15-a453-9d50d62300e8 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:54.747496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730314f-36a4-46a0-bebc-ad6d04c322f3 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:54.750712 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100c6fe-0e68-41fd-b03b-e8ae817c35e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:54.751190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100c6fe-0e68-41fd-b03b-e8ae817c35e7 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:54.751572 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100c6fe-0e68-41fd-b03b-e8ae817c35e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:54.751917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100c6fe-0e68-41fd-b03b-e8ae817c35e7 None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:54.752810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100c6fe-0e68-41fd-b03b-e8ae817c35e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:54.753186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100c6fe-0e68-41fd-b03b-e8ae817c35e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:54.754688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100c6fe-0e68-41fd-b03b-e8ae817c35e7 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:54.769001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 384/768] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:54.770506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff25846-4811-41b2-be59-36129f184b38 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:54.770870 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff25846-4811-41b2-be59-36129f184b38 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:54.771221 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff25846-4811-41b2-be59-36129f184b38 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:54.776520 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 385/769] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:54.776520 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff25846-4811-41b2-be59-36129f184b38 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:54.790435 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfffa46-72da-491e-ba54-6c05251e41c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:54.791418 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfffa46-72da-491e-ba54-6c05251e41c6 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:54.791880 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfffa46-72da-491e-ba54-6c05251e41c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:54.792354 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfffa46-72da-491e-ba54-6c05251e41c6 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:54.793229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfffa46-72da-491e-ba54-6c05251e41c6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:54.793706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfffa46-72da-491e-ba54-6c05251e41c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:54.795270 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfffa46-72da-491e-ba54-6c05251e41c6 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:54.800660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 385/770] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jun 22 11:32:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:54.802157 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6880a-20d0-42ba-b5fc-df6906f8932a None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:54.802641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6880a-20d0-42ba-b5fc-df6906f8932a None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:54.803113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6880a-20d0-42ba-b5fc-df6906f8932a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:54.808578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:54.821001 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-bea6880a-20d0-42ba-b5fc-df6906f8932a None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:54.822509 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-bea6880a-20d0-42ba-b5fc-df6906f8932a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:54.887471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff25846-4811-41b2-be59-36129f184b38 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:54.899453 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 386/771] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:32:54.900645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6880a-20d0-42ba-b5fc-df6906f8932a None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:54.932053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08531418-aea1-4d2f-82f7-c88750638e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:54.932730 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08531418-aea1-4d2f-82f7-c88750638e8d None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:54.933015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08531418-aea1-4d2f-82f7-c88750638e8d None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:54.933294 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08531418-aea1-4d2f-82f7-c88750638e8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:54.934138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08531418-aea1-4d2f-82f7-c88750638e8d None admin] RBAC: Adding query filter params (role.id=96c584428d924fdba9671949f760c406, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=dcd642e290e44e67b56eb3b8361d7af3) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:54.937084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08531418-aea1-4d2f-82f7-c88750638e8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:54.937084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08531418-aea1-4d2f-82f7-c88750638e8d None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:54.946155 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 386/772] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:32:54 2022] GET /identity/v3/role_assignments?user.id=dcd642e290e44e67b56eb3b8361d7af3&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=96c584428d924fdba9671949f760c406 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:54.947213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08531418-aea1-4d2f-82f7-c88750638e8d None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:54.949054 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:55.029976 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6880a-20d0-42ba-b5fc-df6906f8932a None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:55.061909 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:55.070274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08531418-aea1-4d2f-82f7-c88750638e8d None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:55.071180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bfac82-502c-4d0a-a99a-aacaeb5d68bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:55.072021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfac82-502c-4d0a-a99a-aacaeb5d68bd None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:55.072402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfac82-502c-4d0a-a99a-aacaeb5d68bd None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:55.072874 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfac82-502c-4d0a-a99a-aacaeb5d68bd None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:55.073613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bfac82-502c-4d0a-a99a-aacaeb5d68bd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:55.074019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bfac82-502c-4d0a-a99a-aacaeb5d68bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:55.075594 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bfac82-502c-4d0a-a99a-aacaeb5d68bd None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:55.081906 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 387/773] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jun 22 11:32:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:55.092442 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bfac82-502c-4d0a-a99a-aacaeb5d68bd None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:55.120260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb36c9-7663-4b0a-921a-65eb3dc1110f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:55.122021 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb36c9-7663-4b0a-921a-65eb3dc1110f None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:55.122495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb36c9-7663-4b0a-921a-65eb3dc1110f None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:55.122984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb36c9-7663-4b0a-921a-65eb3dc1110f None admin] PATH_INFO: `/v3/services/network` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:55.124190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb36c9-7663-4b0a-921a-65eb3dc1110f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:55.126115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb36c9-7663-4b0a-921a-65eb3dc1110f None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:55.141651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-36eb36c9-7663-4b0a-921a-65eb3dc1110f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 22 11:32:55.141651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:55.141651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:32:55.141651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:32:55.141651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:32:55.141651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:32:55.141651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:32:55.141651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:32:55.141651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 22 11:32:55.141651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:55.141651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:32:55.141651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:55.141651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:55.141651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:55.141651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:55.141651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:55.141651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:55.142717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:55.142717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:55.142717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:55.142717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:55.142717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:55.142717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:55.142717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:32:55.142717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:32:55.142717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:32:55.142717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:32:55.142717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:55.142717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:55.142717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:55.142717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:55.142717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:55.142717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:55.143635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:55.143635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:55.143635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:55.143635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:55.143635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:55.143635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:55.143635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:55.143635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:55.143635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:55.143635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:55.143635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:32:55.143635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:32:55.143635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 22 11:32:55.143635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:55.146268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 387/774] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:54 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:55.152739 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb36c9-7663-4b0a-921a-65eb3dc1110f None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:55.258610 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:55.271664 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bfac82-502c-4d0a-a99a-aacaeb5d68bd None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:55.312655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457d137-b0b3-4baf-90a6-23fd41b727b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:55.313562 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457d137-b0b3-4baf-90a6-23fd41b727b2 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:55.313993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457d137-b0b3-4baf-90a6-23fd41b727b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:55.314478 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457d137-b0b3-4baf-90a6-23fd41b727b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:55.315504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457d137-b0b3-4baf-90a6-23fd41b727b2 None admin] RBAC: Adding query filter params (interface=public, service_id=fbbe0cc595e343ffbed6709f9907a676, region_id=RegionOne) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:55.315985 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457d137-b0b3-4baf-90a6-23fd41b727b2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:55.318077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3457d137-b0b3-4baf-90a6-23fd41b727b2 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:55.327132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 388/775] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jun 22 11:32:55 2022] GET /identity/v3/endpoints?service_id=fbbe0cc595e343ffbed6709f9907a676&interface=public®ion_id=RegionOne => generated 194 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:55.334186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457d137-b0b3-4baf-90a6-23fd41b727b2 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:55.389617 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb36c9-7663-4b0a-921a-65eb3dc1110f None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:55.459368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457d137-b0b3-4baf-90a6-23fd41b727b2 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:55.484087 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b03bc9-f18f-4192-8082-af7e40513c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:55.484776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b03bc9-f18f-4192-8082-af7e40513c41 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:55.485058 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b03bc9-f18f-4192-8082-af7e40513c41 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:55.485335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b03bc9-f18f-4192-8082-af7e40513c41 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:55.486051 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b03bc9-f18f-4192-8082-af7e40513c41 None admin] RBAC: Adding query filter params (name=network) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:55.486358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b03bc9-f18f-4192-8082-af7e40513c41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:55.488296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b03bc9-f18f-4192-8082-af7e40513c41 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:55.498657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f7795-49d1-4f16-972b-01698f815b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:55.499092 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 388/776] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jun 22 11:32:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:55.499784 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f7795-49d1-4f16-972b-01698f815b09 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:55.500214 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f7795-49d1-4f16-972b-01698f815b09 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:55.500637 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f7795-49d1-4f16-972b-01698f815b09 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:55.501489 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f7795-49d1-4f16-972b-01698f815b09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:55.503089 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f7795-49d1-4f16-972b-01698f815b09 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:55.505703 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b03bc9-f18f-4192-8082-af7e40513c41 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:55.510764 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 389/777] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jun 22 11:32:55 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:55.556381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:55.675825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b03bc9-f18f-4192-8082-af7e40513c41 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:55.719802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105a851-9ffe-4548-a111-4d61a93c1f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:55.720562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105a851-9ffe-4548-a111-4d61a93c1f43 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:55.720755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105a851-9ffe-4548-a111-4d61a93c1f43 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:55.721033 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105a851-9ffe-4548-a111-4d61a93c1f43 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:55.721807 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105a851-9ffe-4548-a111-4d61a93c1f43 None admin] RBAC: Adding query filter params (type=network) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:55.722117 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105a851-9ffe-4548-a111-4d61a93c1f43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:55.723509 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105a851-9ffe-4548-a111-4d61a93c1f43 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:55.730238 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 389/778] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jun 22 11:32:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:55.744813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f7795-49d1-4f16-972b-01698f815b09 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:55.768735 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:55.856746 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36237b3-1dd8-4869-ae80-082c2e24febc None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:55.857413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36237b3-1dd8-4869-ae80-082c2e24febc None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:55.857909 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36237b3-1dd8-4869-ae80-082c2e24febc None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:55.860260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 390/779] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:32:55.866404 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbac4cd-e50a-4c3c-be83-077e74f5f8fe None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:55.867138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbac4cd-e50a-4c3c-be83-077e74f5f8fe None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:55.868079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbac4cd-e50a-4c3c-be83-077e74f5f8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:55.888966 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-3dbac4cd-e50a-4c3c-be83-077e74f5f8fe None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:55.890942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-3dbac4cd-e50a-4c3c-be83-077e74f5f8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:55.905632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f7795-49d1-4f16-972b-01698f815b09 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:55.981798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 391/780] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:32:56.007781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbac4cd-e50a-4c3c-be83-077e74f5f8fe None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:56.024110 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2cc21-b0ca-4064-affd-f9cda16b6a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:56.024781 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2cc21-b0ca-4064-affd-f9cda16b6a1a None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:56.025070 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2cc21-b0ca-4064-affd-f9cda16b6a1a None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:56.025352 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2cc21-b0ca-4064-affd-f9cda16b6a1a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:56.026158 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2cc21-b0ca-4064-affd-f9cda16b6a1a None admin] RBAC: Adding query filter params (interface=public, service_id=e7ee756f369346beac01ef12ce5b515d, region_id=RegionOne) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:56.026471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2cc21-b0ca-4064-affd-f9cda16b6a1a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:56.027900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b2cc21-b0ca-4064-affd-f9cda16b6a1a None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:56.038579 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 390/781] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jun 22 11:32:55 2022] GET /identity/v3/endpoints?service_id=e7ee756f369346beac01ef12ce5b515d&interface=public®ion_id=RegionOne => generated 194 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:56.043793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2cc21-b0ca-4064-affd-f9cda16b6a1a None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:56.056976 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:56.133934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbac4cd-e50a-4c3c-be83-077e74f5f8fe None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:56.145046 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:56.166691 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b2cc21-b0ca-4064-affd-f9cda16b6a1a None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:56.187692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db566d-5587-4598-b551-1cf3a01e38d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:56.188859 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db566d-5587-4598-b551-1cf3a01e38d9 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:56.189357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db566d-5587-4598-b551-1cf3a01e38d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:56.190411 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db566d-5587-4598-b551-1cf3a01e38d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:56.204324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ea6c7-2a04-4994-9622-482ff9c6615a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:56.204925 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ea6c7-2a04-4994-9622-482ff9c6615a None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:56.205214 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ea6c7-2a04-4994-9622-482ff9c6615a None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:56.205489 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ea6c7-2a04-4994-9622-482ff9c6615a None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:56.206713 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ea6c7-2a04-4994-9622-482ff9c6615a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:56.207097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db566d-5587-4598-b551-1cf3a01e38d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:56.208137 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ea6c7-2a04-4994-9622-482ff9c6615a None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:56.214047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 391/782] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jun 22 11:32:56 2022] GET /identity/v3/services => generated 772 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:56.215300 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b113261-45ae-4727-a7d7-4d2b05922bd5 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:56.215638 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b113261-45ae-4727-a7d7-4d2b05922bd5 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:56.215923 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b113261-45ae-4727-a7d7-4d2b05922bd5 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:56.217557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-e7db566d-5587-4598-b551-1cf3a01e38d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:32:56.217557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:56.217557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:56.217557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:56.217557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:56.217557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:56.217557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:56.217557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:56.217557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:56.217557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:56.217557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:56.217557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:56.217557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:56.217557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:56.217557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:56.217557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:56.218505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:56.218505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:56.218505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:56.218505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:56.218505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:56.218505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:56.218505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:56.218505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:56.218505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:56.218505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:56.218505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:56.218505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:56.218505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:56.218505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:56.218505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:56.218505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:56.219340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:56.219340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:56.219340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:56.219340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:56.219340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:56.219340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:56.219340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:56.219340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:56.219340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:56.219340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:56.219340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:32:56.219340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:56.221722 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 392/783] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:56.222873 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 392/784] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:32:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:56.227520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62341a3f-8613-464d-84a5-98010cf78dd0 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:56.227520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62341a3f-8613-464d-84a5-98010cf78dd0 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:56.227520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62341a3f-8613-464d-84a5-98010cf78dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:56.229065 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db566d-5587-4598-b551-1cf3a01e38d9 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:56.250164 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-62341a3f-8613-464d-84a5-98010cf78dd0 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:56.252047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-62341a3f-8613-464d-84a5-98010cf78dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:56.339090 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:56.367052 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db566d-5587-4598-b551-1cf3a01e38d9 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:56.408301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa071a42-05c8-4135-baea-d122853e9185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:56.410273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa071a42-05c8-4135-baea-d122853e9185 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:56.410803 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa071a42-05c8-4135-baea-d122853e9185 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:56.411184 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa071a42-05c8-4135-baea-d122853e9185 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:56.412275 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa071a42-05c8-4135-baea-d122853e9185 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:56.412826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa071a42-05c8-4135-baea-d122853e9185 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:56.414613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa071a42-05c8-4135-baea-d122853e9185 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:56.425868 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 393/785] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:56.431603 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa071a42-05c8-4135-baea-d122853e9185 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:56.449116 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 393/786] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:32:56.481983 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62341a3f-8613-464d-84a5-98010cf78dd0 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:56.553940 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa071a42-05c8-4135-baea-d122853e9185 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:56.592547 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963335f-8f9c-4b27-8ce3-02dbfae845fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:56.593746 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963335f-8f9c-4b27-8ce3-02dbfae845fe None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:56.594238 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963335f-8f9c-4b27-8ce3-02dbfae845fe None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:56.594765 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963335f-8f9c-4b27-8ce3-02dbfae845fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:56.596760 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:56.609996 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:56.614079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963335f-8f9c-4b27-8ce3-02dbfae845fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:56.616621 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b963335f-8f9c-4b27-8ce3-02dbfae845fe None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:56.628540 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62341a3f-8613-464d-84a5-98010cf78dd0 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:56.636685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-b963335f-8f9c-4b27-8ce3-02dbfae845fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:56.636685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:56.636685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:56.636685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:56.636685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:56.636685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:56.636685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:56.636685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:56.636685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:56.636685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:56.636685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:56.636685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:56.636685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:56.636685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:56.636685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:56.637657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:56.637657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:56.637657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:56.637657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:56.637657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:56.637657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:56.637657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:56.637657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:56.637657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:56.637657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:56.637657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:56.637657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:56.637657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:56.637657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:56.637657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:56.637657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:56.637657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:56.638749 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:56.638749 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:56.638749 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:56.638749 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:56.640882 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 394/787] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:56.650063 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963335f-8f9c-4b27-8ce3-02dbfae845fe None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:56.683782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e118c9-4091-4283-9434-3d09acc93a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:56.685138 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e118c9-4091-4283-9434-3d09acc93a7b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:56.685987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e118c9-4091-4283-9434-3d09acc93a7b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:56.686546 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e118c9-4091-4283-9434-3d09acc93a7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:56.701902 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e118c9-4091-4283-9434-3d09acc93a7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:56.713265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-98e118c9-4091-4283-9434-3d09acc93a7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:56.713265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:56.713265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:56.713265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:56.713265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:56.713265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:56.713265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:56.713265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:56.713265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:56.713265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:56.713265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:56.713265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:56.713265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:56.713265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:56.713265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:56.713265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:56.714235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:56.714235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:56.714235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:56.714235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:56.714235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:56.714235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:56.714235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:56.714235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:56.714235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:56.714235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:56.714235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:56.714235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:56.714235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:56.714235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:56.714235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:56.714235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:56.715094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:56.715094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:56.715094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:56.715094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:56.715094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:56.715094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:56.715094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:56.715094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:56.715094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:56.715094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:56.715094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:32:56.715094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:56.717788 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 394/788] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:32:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:56.724464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e118c9-4091-4283-9434-3d09acc93a7b None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:56.844270 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e118c9-4091-4283-9434-3d09acc93a7b None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:56.845480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b963335f-8f9c-4b27-8ce3-02dbfae845fe None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:56.885000 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438f09a-64d5-4c11-90a0-8b4deadbeaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:56.885937 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438f09a-64d5-4c11-90a0-8b4deadbeaec None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:56.886376 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438f09a-64d5-4c11-90a0-8b4deadbeaec None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:56.886804 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438f09a-64d5-4c11-90a0-8b4deadbeaec None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:56.887857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438f09a-64d5-4c11-90a0-8b4deadbeaec None admin] RBAC: Adding query filter params (name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:56.888314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438f09a-64d5-4c11-90a0-8b4deadbeaec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:56.890126 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438f09a-64d5-4c11-90a0-8b4deadbeaec None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:56.893125 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:56.903475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 395/789] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:32:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:56.907507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae204e7-04aa-43c7-95eb-e8b53929b9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:56.907507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae204e7-04aa-43c7-95eb-e8b53929b9f8 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:56.907507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae204e7-04aa-43c7-95eb-e8b53929b9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:56.907507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae204e7-04aa-43c7-95eb-e8b53929b9f8 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:56.907905 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae204e7-04aa-43c7-95eb-e8b53929b9f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:56.907905 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae204e7-04aa-43c7-95eb-e8b53929b9f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:56.907905 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae204e7-04aa-43c7-95eb-e8b53929b9f8 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:56.910954 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438f09a-64d5-4c11-90a0-8b4deadbeaec None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:56.921133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 395/790] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:32:56.928729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae204e7-04aa-43c7-95eb-e8b53929b9f8 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:56.995760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:57.048225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae204e7-04aa-43c7-95eb-e8b53929b9f8 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:57.089182 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6499b-bf12-4a2a-b2ea-71959eb8c59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:57.090218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6499b-bf12-4a2a-b2ea-71959eb8c59e None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:57.090875 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6499b-bf12-4a2a-b2ea-71959eb8c59e None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:57.092087 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6499b-bf12-4a2a-b2ea-71959eb8c59e None admin] PATH_INFO: `/v3/users/nova` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:57.102360 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6499b-bf12-4a2a-b2ea-71959eb8c59e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:57.108890 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6499b-bf12-4a2a-b2ea-71959eb8c59e None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:57.115516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438f09a-64d5-4c11-90a0-8b4deadbeaec None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:57.143326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-12e6499b-bf12-4a2a-b2ea-71959eb8c59e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 22 11:32:57.143326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:57.143326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:57.143326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:57.143326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:57.143326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:57.143326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:57.143326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:57.143326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:57.143326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:57.143326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:57.143326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:57.143326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:57.143326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:57.143326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:57.143326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:57.144357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:57.144357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:57.144357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:57.144357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:57.144357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:57.144357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:57.144357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:57.144357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:57.144357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:57.144357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:57.144357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:57.144357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:57.144357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:57.144357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:57.144357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:57.144357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:57.144357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:57.144357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:57.146445 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:57.146445 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:57.146445 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:57.146445 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:57.146445 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:57.146445 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:57.146445 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:57.146445 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:57.146445 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 22 11:32:57.146445 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:57.146445 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 396/791] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jun 22 11:32:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:57.161275 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6499b-bf12-4a2a-b2ea-71959eb8c59e None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:57.163687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25adc068-7f14-42a7-853e-9ef3cc191999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:57.164678 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adc068-7f14-42a7-853e-9ef3cc191999 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:57.165108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adc068-7f14-42a7-853e-9ef3cc191999 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:57.165530 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25adc068-7f14-42a7-853e-9ef3cc191999 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:57.177234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25adc068-7f14-42a7-853e-9ef3cc191999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:57.178867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25adc068-7f14-42a7-853e-9ef3cc191999 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:57.196828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-25adc068-7f14-42a7-853e-9ef3cc191999 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:57.196828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:57.196828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:57.196828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:57.196828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:57.196828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:57.196828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:57.196828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:57.196828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:57.196828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:57.196828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:57.196828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:57.196828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:57.196828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:57.196828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:57.197758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:57.197758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:57.197758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:57.197758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:57.197758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:57.197758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:57.197758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:57.197758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:57.197758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:57.197758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:57.197758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:57.197758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:57.197758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:57.197758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:57.197758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:57.197758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:57.197758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:57.198640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:57.198640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:57.198640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:57.198640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:57.202430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 396/792] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:57.208208 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25adc068-7f14-42a7-853e-9ef3cc191999 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:57.309299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:57.372129 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6499b-bf12-4a2a-b2ea-71959eb8c59e None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:57.396513 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:57.408081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25adc068-7f14-42a7-853e-9ef3cc191999 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:57.430377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122b870-fc3d-4156-ab85-8713a5b2015a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:57.431322 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b870-fc3d-4156-ab85-8713a5b2015a None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:57.431639 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b870-fc3d-4156-ab85-8713a5b2015a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:57.431917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b870-fc3d-4156-ab85-8713a5b2015a None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:57.433015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122b870-fc3d-4156-ab85-8713a5b2015a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:57.433331 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122b870-fc3d-4156-ab85-8713a5b2015a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:57.435573 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122b870-fc3d-4156-ab85-8713a5b2015a None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:57.446974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b15c4b-f9f5-4cd0-8a4f-41d09d777b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:57.447933 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b15c4b-f9f5-4cd0-8a4f-41d09d777b80 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:57.448701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b15c4b-f9f5-4cd0-8a4f-41d09d777b80 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:57.449269 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b15c4b-f9f5-4cd0-8a4f-41d09d777b80 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:57.451166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b15c4b-f9f5-4cd0-8a4f-41d09d777b80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:57.451683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b15c4b-f9f5-4cd0-8a4f-41d09d777b80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:57.453252 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b15c4b-f9f5-4cd0-8a4f-41d09d777b80 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:57.464769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 397/793] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jun 22 11:32:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:57.464769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa08b9-0f7a-48c4-915f-840f6eb26f22 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:57.464769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa08b9-0f7a-48c4-915f-840f6eb26f22 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:57.464769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa08b9-0f7a-48c4-915f-840f6eb26f22 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:57.464769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 398/794] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:57.469928 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa08b9-0f7a-48c4-915f-840f6eb26f22 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:57.476259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 397/795] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:57.477855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418604b-466b-4c8d-b57f-9048aff2ff33 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:57.478292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418604b-466b-4c8d-b57f-9048aff2ff33 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:57.478751 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418604b-466b-4c8d-b57f-9048aff2ff33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:57.500472 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-4418604b-466b-4c8d-b57f-9048aff2ff33 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:57.501921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-4418604b-466b-4c8d-b57f-9048aff2ff33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:57.604787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa08b9-0f7a-48c4-915f-840f6eb26f22 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:57.623027 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 398/796] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:32:57.623943 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418604b-466b-4c8d-b57f-9048aff2ff33 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:57.645513 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e406b80-182e-432a-8b95-db5191265b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:57.646428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e406b80-182e-432a-8b95-db5191265b45 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:57.646860 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e406b80-182e-432a-8b95-db5191265b45 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:57.647313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e406b80-182e-432a-8b95-db5191265b45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:57.689906 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:57.691286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:57.724760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e406b80-182e-432a-8b95-db5191265b45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:57.726450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e406b80-182e-432a-8b95-db5191265b45 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:57.758612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418604b-466b-4c8d-b57f-9048aff2ff33 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:57.799457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc900cf-67f6-42f4-9253-dcbee6ccc015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:57.800712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc900cf-67f6-42f4-9253-dcbee6ccc015 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:57.801147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc900cf-67f6-42f4-9253-dcbee6ccc015 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:57.801655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc900cf-67f6-42f4-9253-dcbee6ccc015 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:57.811151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc900cf-67f6-42f4-9253-dcbee6ccc015 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:57.812991 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc900cf-67f6-42f4-9253-dcbee6ccc015 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:57.829833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-6cc900cf-67f6-42f4-9253-dcbee6ccc015 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 22 11:32:57.829833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:57.829833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:57.829833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:57.829833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:57.829833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:57.829833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:57.829833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:57.829833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:57.829833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:57.829833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:57.829833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:57.829833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:57.829833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:57.829833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:57.829833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:57.831527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:57.831527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:57.831527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:57.831527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:57.831527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:57.831527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:57.831527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:57.831527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:57.831527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:57.831527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:57.831527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:57.831527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:57.831527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:57.831527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:57.831527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:57.831527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:57.831527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:57.831527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:57.832483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:57.832483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:57.832483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:57.832483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:57.832483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:57.832483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:57.832483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:57.832483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:57.832483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 22 11:32:57.832483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:57.839333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 399/798] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jun 22 11:32:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:57.840450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc900cf-67f6-42f4-9253-dcbee6ccc015 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:57.841762 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 399/798] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:32:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 22 11:32:57.842671 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e406b80-182e-432a-8b95-db5191265b45 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:57.914846 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:57.947737 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e406b80-182e-432a-8b95-db5191265b45 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:57.959167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:57.963764 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc900cf-67f6-42f4-9253-dcbee6ccc015 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:57.986165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029069c-9b0e-48bf-8c9e-83d43ce996b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:57.986946 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029069c-9b0e-48bf-8c9e-83d43ce996b4 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:57.987255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029069c-9b0e-48bf-8c9e-83d43ce996b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:57.987669 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029069c-9b0e-48bf-8c9e-83d43ce996b4 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:57.988505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0029069c-9b0e-48bf-8c9e-83d43ce996b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:57.988815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0029069c-9b0e-48bf-8c9e-83d43ce996b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:57.990234 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0029069c-9b0e-48bf-8c9e-83d43ce996b4 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:58.005221 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f4ff4-ee06-401e-b774-5acce592494d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:58.006011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f4ff4-ee06-401e-b774-5acce592494d None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:58.006297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f4ff4-ee06-401e-b774-5acce592494d None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:58.006575 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f4ff4-ee06-401e-b774-5acce592494d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:58.010396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 400/799] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jun 22 11:32:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:58.011600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029069c-9b0e-48bf-8c9e-83d43ce996b4 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:58.021616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f4ff4-ee06-401e-b774-5acce592494d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:32:58.030669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-3f7f4ff4-ee06-401e-b774-5acce592494d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:32:58.030669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:58.030669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:58.030669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:58.030669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:58.030669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:58.030669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:58.030669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:58.030669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:58.030669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:58.030669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:58.030669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:58.030669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:32:58.030669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:32:58.030669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:32:58.030669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:32:58.031643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:32:58.031643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:32:58.031643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:32:58.031643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:32:58.031643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:58.031643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:58.031643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:58.031643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:58.031643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:58.031643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:58.031643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:58.031643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:58.031643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:58.031643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:58.031643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:58.031643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:58.032599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:58.032599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:58.032599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:58.032599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:58.032599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:32:58.032599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:32:58.032599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:32:58.032599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:32:58.032599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:32:58.032599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:32:58.032599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:32:58.032599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:58.032599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 400/800] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:32:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:58.033287 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74fe7a-9672-4f00-a311-1f7a3ca38169 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:58.033432 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74fe7a-9672-4f00-a311-1f7a3ca38169 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:58.033712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b74fe7a-9672-4f00-a311-1f7a3ca38169 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:58.035153 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 401/801] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:58.039670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74fe7a-9672-4f00-a311-1f7a3ca38169 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:58.126593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029069c-9b0e-48bf-8c9e-83d43ce996b4 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:58.155612 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:58.155895 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b74fe7a-9672-4f00-a311-1f7a3ca38169 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:58.165638 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c209c6-1a85-4c83-83a9-1cae0e3d044a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:58.166999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c209c6-1a85-4c83-83a9-1cae0e3d044a None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:58.167792 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c209c6-1a85-4c83-83a9-1cae0e3d044a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:58.168513 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c209c6-1a85-4c83-83a9-1cae0e3d044a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:58.182498 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c209c6-1a85-4c83-83a9-1cae0e3d044a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:58.184800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c209c6-1a85-4c83-83a9-1cae0e3d044a None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:58.191117 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdfa76-5c09-4dbb-8ce5-d5250914337b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:58.191942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdfa76-5c09-4dbb-8ce5-d5250914337b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:58.192227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdfa76-5c09-4dbb-8ce5-d5250914337b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:58.192503 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdfa76-5c09-4dbb-8ce5-d5250914337b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:58.203051 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fdfa76-5c09-4dbb-8ce5-d5250914337b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:58.204537 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fdfa76-5c09-4dbb-8ce5-d5250914337b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:58.207301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-77c209c6-1a85-4c83-83a9-1cae0e3d044a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:58.207301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:58.207301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:58.207301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:58.207301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:58.207301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:58.207301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:58.207301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:58.207301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:58.207301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:58.207301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:58.207301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:58.207301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:58.207301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:58.207301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:58.208372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:58.208372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:58.208372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:58.208372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:58.208372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:58.208372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:58.208372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:58.208372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:58.208372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:58.208372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:58.208372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:58.208372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:58.208372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:58.208372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:58.208372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:58.208372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:58.208372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:58.209443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:58.209443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:58.209443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:58.209443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:58.211741 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 401/802] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:58.212679 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c209c6-1a85-4c83-83a9-1cae0e3d044a None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:58.224822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-e3fdfa76-5c09-4dbb-8ce5-d5250914337b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:58.224822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:58.224822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:58.224822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:58.224822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:58.224822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:58.224822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:58.224822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:58.224822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:58.224822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:58.224822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:58.224822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:58.224822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:58.224822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:58.224822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:58.225952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:58.225952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:58.225952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:58.225952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:58.225952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:58.225952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:58.225952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:58.225952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:58.225952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:58.225952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:58.225952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:58.225952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:58.225952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:58.225952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:58.225952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:58.225952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:58.225952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:58.226887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:58.226887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:58.226887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:58.226887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:58.226887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 402/803] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:58.227260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985606a-638e-4a87-be23-acb635fa37c7 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:58.227548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985606a-638e-4a87-be23-acb635fa37c7 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:58.227828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985606a-638e-4a87-be23-acb635fa37c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:58.244804 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f985606a-638e-4a87-be23-acb635fa37c7 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:58.245994 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f985606a-638e-4a87-be23-acb635fa37c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:58.254083 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:58.320650 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 403/804] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:32:58.321472 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985606a-638e-4a87-be23-acb635fa37c7 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:58.340127 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c209c6-1a85-4c83-83a9-1cae0e3d044a None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:58.379116 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bcc86-aea5-400e-bc50-6c4988a95a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:58.379819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bcc86-aea5-400e-bc50-6c4988a95a34 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:58.380104 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bcc86-aea5-400e-bc50-6c4988a95a34 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:58.380391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bcc86-aea5-400e-bc50-6c4988a95a34 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:58.381132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bcc86-aea5-400e-bc50-6c4988a95a34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:58.381437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bcc86-aea5-400e-bc50-6c4988a95a34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:58.382796 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bcc86-aea5-400e-bc50-6c4988a95a34 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:58.391654 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 402/805] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:32:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:58.392317 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bcc86-aea5-400e-bc50-6c4988a95a34 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:58.438270 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:58.448826 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985606a-638e-4a87-be23-acb635fa37c7 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:58.487757 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673684d8-5604-425e-a0b6-c19e3d211c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:58.489921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673684d8-5604-425e-a0b6-c19e3d211c03 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:58.490204 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673684d8-5604-425e-a0b6-c19e3d211c03 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:58.490475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673684d8-5604-425e-a0b6-c19e3d211c03 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:58.491201 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673684d8-5604-425e-a0b6-c19e3d211c03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:58.491544 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673684d8-5604-425e-a0b6-c19e3d211c03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:58.492936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673684d8-5604-425e-a0b6-c19e3d211c03 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:58.505103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bcc86-aea5-400e-bc50-6c4988a95a34 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:58.509724 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 404/806] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:58.509724 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673684d8-5604-425e-a0b6-c19e3d211c03 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:58.518433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:58.550828 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da5387-2a5d-49b0-a6ce-93149598e479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:58.551521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da5387-2a5d-49b0-a6ce-93149598e479 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:58.551803 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da5387-2a5d-49b0-a6ce-93149598e479 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:58.552076 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da5387-2a5d-49b0-a6ce-93149598e479 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:58.552791 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da5387-2a5d-49b0-a6ce-93149598e479 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:58.553093 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da5387-2a5d-49b0-a6ce-93149598e479 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:58.554455 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da5387-2a5d-49b0-a6ce-93149598e479 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:58.571282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 403/807] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:58.571282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da5387-2a5d-49b0-a6ce-93149598e479 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:58.622756 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673684d8-5604-425e-a0b6-c19e3d211c03 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:58.659687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f832d-5381-42de-bdea-62a19d2b381b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:58.660495 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f832d-5381-42de-bdea-62a19d2b381b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:58.660775 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f832d-5381-42de-bdea-62a19d2b381b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:58.661044 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f832d-5381-42de-bdea-62a19d2b381b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:58.661687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f832d-5381-42de-bdea-62a19d2b381b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:58.663077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f832d-5381-42de-bdea-62a19d2b381b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:58.673537 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da5387-2a5d-49b0-a6ce-93149598e479 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:58.675636 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:58.675796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-504f832d-5381-42de-bdea-62a19d2b381b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 22 11:32:58.675796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:58.675796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:32:58.675796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:32:58.675796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:32:58.675796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:32:58.675796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:32:58.675796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:32:58.675796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 22 11:32:58.675796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:58.675796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:32:58.675796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:58.675796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:58.675796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:58.675796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:58.675796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:58.676870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:58.676870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:58.676870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:58.676870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:58.676870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:58.676870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:58.676870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:58.676870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:32:58.676870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:32:58.676870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:32:58.676870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:32:58.676870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:58.676870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:58.676870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:58.676870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:58.676870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:58.677709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:58.677709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:58.677709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:58.677709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:58.677709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:58.677709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:58.677709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:58.677709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:58.677709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:58.677709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:58.677709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:58.677709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:32:58.677709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:32:58.677709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 22 11:32:58.677709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:58.677709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 405/808] 10.222.0.36 () {56 vars in 1129 bytes} [Wed Jun 22 11:32:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:58.678548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f832d-5381-42de-bdea-62a19d2b381b None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:58.720890 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70aeef4-4031-4c7f-ad40-960165fe4383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:58.722184 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70aeef4-4031-4c7f-ad40-960165fe4383 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:58.722465 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70aeef4-4031-4c7f-ad40-960165fe4383 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:58.722740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70aeef4-4031-4c7f-ad40-960165fe4383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:58.733705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70aeef4-4031-4c7f-ad40-960165fe4383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:58.735162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70aeef4-4031-4c7f-ad40-960165fe4383 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:58.748887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:58.753974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-c70aeef4-4031-4c7f-ad40-960165fe4383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:58.753974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:58.753974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:58.753974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:58.753974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:58.753974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:58.753974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:58.753974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:58.753974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:58.753974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:58.753974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:58.753974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:58.753974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:32:58.753974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:32:58.753974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:32:58.755417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:32:58.755417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:58.755417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:58.755417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:58.755417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:58.755417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:58.755417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:58.755417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:58.755417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:58.755417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:58.755417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:58.755417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:58.755417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:58.755417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:58.755417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:58.755417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:58.755417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:58.756429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:32:58.756429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:32:58.756429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:32:58.756429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:58.756429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 404/809] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:58.756429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70aeef4-4031-4c7f-ad40-960165fe4383 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:58.789561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f832d-5381-42de-bdea-62a19d2b381b None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:58.825929 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e5245-9a83-42db-aee6-356f1b1923f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:58.827116 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e5245-9a83-42db-aee6-356f1b1923f9 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:58.827265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e5245-9a83-42db-aee6-356f1b1923f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:58.827573 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e5245-9a83-42db-aee6-356f1b1923f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:58.835726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e5245-9a83-42db-aee6-356f1b1923f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:58.838032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e5245-9a83-42db-aee6-356f1b1923f9 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:58.856553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-465e5245-9a83-42db-aee6-356f1b1923f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:58.856553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:58.856553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:58.856553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:58.856553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:58.856553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:58.856553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:58.856553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:58.856553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:58.856553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:58.856553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:58.856553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:58.856553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:58.856553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:58.856553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:58.857857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:58.857857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:58.857857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:58.857857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:58.857857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:58.857857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:58.857857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:58.857857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:58.857857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:58.857857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:58.857857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:58.857857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:58.857857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:58.857857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:58.857857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:58.857857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:58.857857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:58.858775 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:58.858775 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:58.858775 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:58.858775 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:58.858775 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:58.858775 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:58.858775 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:58.858775 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:58.858775 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 406/810] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:58.858775 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e5245-9a83-42db-aee6-356f1b1923f9 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:58.869002 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70aeef4-4031-4c7f-ad40-960165fe4383 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:58.904930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f7e90-a123-4108-ba46-e0dd5bde2767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:58.905720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f7e90-a123-4108-ba46-e0dd5bde2767 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:58.906003 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f7e90-a123-4108-ba46-e0dd5bde2767 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:58.906287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f7e90-a123-4108-ba46-e0dd5bde2767 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:58.912954 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f7e90-a123-4108-ba46-e0dd5bde2767 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:58.914383 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f7e90-a123-4108-ba46-e0dd5bde2767 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:58.929289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-8b2f7e90-a123-4108-ba46-e0dd5bde2767 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:58.929289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:58.929289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:58.929289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:58.929289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:58.929289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:58.929289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:58.929289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:58.929289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:58.929289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:58.929289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:58.929289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:58.929289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:32:58.929289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:32:58.929289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:32:58.930814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:32:58.930814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:58.930814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:58.930814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:58.930814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:58.930814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:58.930814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:58.930814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:58.930814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:58.930814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:58.930814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:58.930814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:58.930814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:58.930814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:58.930814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:58.930814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:58.930814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:58.931782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:32:58.931782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:32:58.931782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:32:58.931782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:32:58.931782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:32:58.931782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:32:58.931782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:32:58.931782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:58.931782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 405/811] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:58.933058 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eaaa22-93c0-441f-a645-269bcc384bde None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:58.933355 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eaaa22-93c0-441f-a645-269bcc384bde None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:58.933638 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eaaa22-93c0-441f-a645-269bcc384bde None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:58.934955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 406/812] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:32:58.936174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eaaa22-93c0-441f-a645-269bcc384bde None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:58.953498 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:58.962818 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e5245-9a83-42db-aee6-356f1b1923f9 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:58.989119 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:58.998142 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8614b54-eccc-4eee-a8b8-8e65de37b7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:58.998807 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8614b54-eccc-4eee-a8b8-8e65de37b7b8 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:58.999087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8614b54-eccc-4eee-a8b8-8e65de37b7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:58.999378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8614b54-eccc-4eee-a8b8-8e65de37b7b8 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:59.000087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8614b54-eccc-4eee-a8b8-8e65de37b7b8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:59.000437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8614b54-eccc-4eee-a8b8-8e65de37b7b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:59.001753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8614b54-eccc-4eee-a8b8-8e65de37b7b8 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:59.006706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 407/813] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jun 22 11:32:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:59.007503 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8614b54-eccc-4eee-a8b8-8e65de37b7b8 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:59.043339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eaaa22-93c0-441f-a645-269bcc384bde None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:59.076839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0138a7-581e-4cad-9e92-fa37147d0925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:59.077496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0138a7-581e-4cad-9e92-fa37147d0925 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:59.077775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0138a7-581e-4cad-9e92-fa37147d0925 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:59.078047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0138a7-581e-4cad-9e92-fa37147d0925 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:59.078756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0138a7-581e-4cad-9e92-fa37147d0925 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:59.079058 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0138a7-581e-4cad-9e92-fa37147d0925 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:59.080449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0138a7-581e-4cad-9e92-fa37147d0925 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:59.092603 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 407/814] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:59.093247 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0138a7-581e-4cad-9e92-fa37147d0925 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:59.115617 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8614b54-eccc-4eee-a8b8-8e65de37b7b8 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:59.150310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf32f39-2eee-4fe3-b023-a8526c2a5ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:59.150962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf32f39-2eee-4fe3-b023-a8526c2a5ef1 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:59.151245 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf32f39-2eee-4fe3-b023-a8526c2a5ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:59.151541 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf32f39-2eee-4fe3-b023-a8526c2a5ef1 None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:59.152320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf32f39-2eee-4fe3-b023-a8526c2a5ef1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:59.152623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf32f39-2eee-4fe3-b023-a8526c2a5ef1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:59.154054 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf32f39-2eee-4fe3-b023-a8526c2a5ef1 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:59.168822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 408/815] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:59.170073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b6d16-2c51-4a9e-a97b-d5ba0a951cd8 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:59.170361 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b6d16-2c51-4a9e-a97b-d5ba0a951cd8 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:59.170639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b6d16-2c51-4a9e-a97b-d5ba0a951cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:59.175942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:59.189912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-111b6d16-2c51-4a9e-a97b-d5ba0a951cd8 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:32:59.191697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-111b6d16-2c51-4a9e-a97b-d5ba0a951cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:32:59.201394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0138a7-581e-4cad-9e92-fa37147d0925 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:59.234305 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b64494-725b-4e99-ab43-3670f4abe9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:59.234968 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b64494-725b-4e99-ab43-3670f4abe9ac None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:59.235255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b64494-725b-4e99-ab43-3670f4abe9ac None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:59.235568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b64494-725b-4e99-ab43-3670f4abe9ac None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:59.236362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b64494-725b-4e99-ab43-3670f4abe9ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:59.236670 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b64494-725b-4e99-ab43-3670f4abe9ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:59.238613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b64494-725b-4e99-ab43-3670f4abe9ac None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:59.240759 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:59.251831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 408/816] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:32:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:59.252566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b64494-725b-4e99-ab43-3670f4abe9ac None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:59.263667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 409/817] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:32:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:32:59.264518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b6d16-2c51-4a9e-a97b-d5ba0a951cd8 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:59.352796 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b64494-725b-4e99-ab43-3670f4abe9ac None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:59.365230 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b6d16-2c51-4a9e-a97b-d5ba0a951cd8 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:59.386223 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c1422-a7fc-4459-a321-13a527e9cf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:59.386907 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c1422-a7fc-4459-a321-13a527e9cf89 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:59.387187 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c1422-a7fc-4459-a321-13a527e9cf89 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:59.387489 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c1422-a7fc-4459-a321-13a527e9cf89 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:59.388188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c1422-a7fc-4459-a321-13a527e9cf89 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:59.388488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c1422-a7fc-4459-a321-13a527e9cf89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:59.389842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c1422-a7fc-4459-a321-13a527e9cf89 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:59.391304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:59.401614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 409/818] 10.222.0.36 () {56 vars in 1124 bytes} [Wed Jun 22 11:32:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:59.401614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c1422-a7fc-4459-a321-13a527e9cf89 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:59.401860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe570941-ee73-4b5e-b015-311de5782b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:59.401860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe570941-ee73-4b5e-b015-311de5782b8f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:59.401860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe570941-ee73-4b5e-b015-311de5782b8f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:59.401860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe570941-ee73-4b5e-b015-311de5782b8f None admin] PATH_INFO: `/v3/users/swift` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:59.408132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe570941-ee73-4b5e-b015-311de5782b8f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:59.410151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe570941-ee73-4b5e-b015-311de5782b8f None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:59.425285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-fe570941-ee73-4b5e-b015-311de5782b8f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 22 11:32:59.425285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:59.425285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:59.425285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:59.425285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:59.425285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:59.425285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:59.425285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:59.425285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:59.425285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:59.425285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:59.425285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:59.425285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:32:59.425285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:32:59.425285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:32:59.425285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:32:59.426477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:59.426477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:59.426477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:32:59.426477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:59.426477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:32:59.426477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:32:59.426477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:59.426477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:59.426477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:59.426477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:59.426477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:59.426477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:59.426477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:59.426477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:59.426477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:59.426477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:59.426477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:59.426477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:59.427646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:59.427646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:59.427646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:32:59.427646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:32:59.427646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:32:59.427646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:32:59.427646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:32:59.427646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:32:59.427646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 22 11:32:59.427646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:32:59.428551 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 410/819] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:32:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:59.429357 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe570941-ee73-4b5e-b015-311de5782b8f None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:59.476487 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:59.519755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c1422-a7fc-4459-a321-13a527e9cf89 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:59.548335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe570941-ee73-4b5e-b015-311de5782b8f None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:59.561565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfb656-a095-49cc-9ac8-0d34c2047ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:59.562384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfb656-a095-49cc-9ac8-0d34c2047ae5 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:59.562676 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfb656-a095-49cc-9ac8-0d34c2047ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:59.563302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebfb656-a095-49cc-9ac8-0d34c2047ae5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:59.564448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfb656-a095-49cc-9ac8-0d34c2047ae5 None admin] RBAC: Adding query filter params (role.id=96c584428d924fdba9671949f760c406, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=479a376dcac14c509c1c55f0046d88cf) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:32:59.564761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfb656-a095-49cc-9ac8-0d34c2047ae5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:59.567187 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebfb656-a095-49cc-9ac8-0d34c2047ae5 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:59.574668 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 410/820] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:32:59 2022] GET /identity/v3/role_assignments?user.id=479a376dcac14c509c1c55f0046d88cf&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=96c584428d924fdba9671949f760c406 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:32:59.575603 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfb656-a095-49cc-9ac8-0d34c2047ae5 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:59.599025 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d22fe-90e5-41f2-b3a1-fc9a98fcb7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:59.601166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d22fe-90e5-41f2-b3a1-fc9a98fcb7dc None admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:59.601293 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d22fe-90e5-41f2-b3a1-fc9a98fcb7dc None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:59.601569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d22fe-90e5-41f2-b3a1-fc9a98fcb7dc None admin] PATH_INFO: `/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/955c44ffe1bd426e900c7405df44d1cf/roles/64c2d69f15ca458c8267c1acfd42d9fc` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:59.620754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d22fe-90e5-41f2-b3a1-fc9a98fcb7dc None admin] RBAC: Authorizing `identity:create_grant(project_id=47d2bb094c9843c9a7726f1d672d2433, user_id=955c44ffe1bd426e900c7405df44d1cf, role_id=64c2d69f15ca458c8267c1acfd42d9fc)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:59.622423 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d22fe-90e5-41f2-b3a1-fc9a98fcb7dc None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:59.647385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 411/821] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:32:59 2022] PUT /identity/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/955c44ffe1bd426e900c7405df44d1cf/roles/64c2d69f15ca458c8267c1acfd42d9fc => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:32:59.648260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d22fe-90e5-41f2-b3a1-fc9a98fcb7dc None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:59.684852 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:59.765309 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebfb656-a095-49cc-9ac8-0d34c2047ae5 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:59.824439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4703b1-9002-4917-942f-7f3e57403b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:59.825537 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4703b1-9002-4917-942f-7f3e57403b57 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:59.825821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4703b1-9002-4917-942f-7f3e57403b57 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:59.826097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4703b1-9002-4917-942f-7f3e57403b57 None admin] PATH_INFO: `/v3/services/network` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:59.826754 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4703b1-9002-4917-942f-7f3e57403b57 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:59.828620 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4703b1-9002-4917-942f-7f3e57403b57 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:59.832855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:32:59.842393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-dd4703b1-9002-4917-942f-7f3e57403b57 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 22 11:32:59.842393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:59.842393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:32:59.842393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:32:59.842393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:32:59.842393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:32:59.842393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:32:59.842393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:32:59.842393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 22 11:32:59.842393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:59.842393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:32:59.842393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:59.842393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:32:59.842393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:32:59.842393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:32:59.842393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:32:59.842393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:32:59.843495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:32:59.843495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:32:59.843495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:32:59.843495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:32:59.843495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:32:59.843495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:32:59.843495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:32:59.843495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:32:59.843495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:32:59.843495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:32:59.843495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:32:59.843495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:32:59.843495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:32:59.843495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:32:59.843495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:32:59.843495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:32:59.844337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:32:59.844337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:32:59.844337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:32:59.844337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:32:59.844337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:32:59.844337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:32:59.844337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:32:59.844337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:32:59.844337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:32:59.844337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:32:59.844337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:32:59.844337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:32:59.844337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 22 11:32:59.844337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:32:59.844337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 411/822] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:32:59 2022] GET /identity/v3/services/network => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:32:59.845155 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4703b1-9002-4917-942f-7f3e57403b57 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:59.849180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d22fe-90e5-41f2-b3a1-fc9a98fcb7dc None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:32:59.892992 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310326e-b3ee-4959-b50f-3d9b9a8ef144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:32:59.893991 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310326e-b3ee-4959-b50f-3d9b9a8ef144 None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:32:59.894423 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310326e-b3ee-4959-b50f-3d9b9a8ef144 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:32:59.894847 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310326e-b3ee-4959-b50f-3d9b9a8ef144 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:32:59.895797 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310326e-b3ee-4959-b50f-3d9b9a8ef144 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:32:59.897392 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310326e-b3ee-4959-b50f-3d9b9a8ef144 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:32:59.929696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 412/823] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jun 22 11:32:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:32:59.932091 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310326e-b3ee-4959-b50f-3d9b9a8ef144 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:32:59.980837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4703b1-9002-4917-942f-7f3e57403b57 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:00.020614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:00.038930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c23954-f6df-4a9d-91b0-8281f2f45317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:00.040134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c23954-f6df-4a9d-91b0-8281f2f45317 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:00.040623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c23954-f6df-4a9d-91b0-8281f2f45317 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:00.041106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c23954-f6df-4a9d-91b0-8281f2f45317 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:00.043100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c23954-f6df-4a9d-91b0-8281f2f45317 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:00.043682 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c23954-f6df-4a9d-91b0-8281f2f45317 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:00.046807 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c23954-f6df-4a9d-91b0-8281f2f45317 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:00.072476 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 412/824] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jun 22 11:32:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:00.079670 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c23954-f6df-4a9d-91b0-8281f2f45317 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:00.101983 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310326e-b3ee-4959-b50f-3d9b9a8ef144 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:00.176855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a18c29-dede-4dcb-b72e-51124c9f0bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:00.177637 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a18c29-dede-4dcb-b72e-51124c9f0bdb None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:00.177878 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a18c29-dede-4dcb-b72e-51124c9f0bdb None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:00.178213 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a18c29-dede-4dcb-b72e-51124c9f0bdb None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:00.179013 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a18c29-dede-4dcb-b72e-51124c9f0bdb None admin] RBAC: Adding query filter params (name=network) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:00.179417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a18c29-dede-4dcb-b72e-51124c9f0bdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:00.180915 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a18c29-dede-4dcb-b72e-51124c9f0bdb None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:00.187305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 413/825] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jun 22 11:32:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:00.194060 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a18c29-dede-4dcb-b72e-51124c9f0bdb None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:00.213842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:00.279825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c23954-f6df-4a9d-91b0-8281f2f45317 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:00.320589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3e56e-fc96-4a54-a04f-04de4ed4e5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:00.325931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3e56e-fc96-4a54-a04f-04de4ed4e5fe None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:00.326443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3e56e-fc96-4a54-a04f-04de4ed4e5fe None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:00.327004 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3e56e-fc96-4a54-a04f-04de4ed4e5fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:00.346108 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3e56e-fc96-4a54-a04f-04de4ed4e5fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:00.348163 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3e56e-fc96-4a54-a04f-04de4ed4e5fe None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:00.377496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-70b3e56e-fc96-4a54-a04f-04de4ed4e5fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:00.377496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:00.377496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:00.377496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:00.377496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:00.377496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:00.377496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:00.377496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:00.377496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:00.377496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:00.377496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:00.377496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:00.377496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:33:00.377496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:33:00.377496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:33:00.379000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:33:00.379000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:00.379000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:00.379000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:00.379000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:00.379000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:00.379000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:00.379000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:00.379000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:00.379000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:00.379000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:00.379000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:00.379000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:00.379000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:00.379000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:00.379000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:00.379000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:00.379955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:33:00.379955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:33:00.379955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:00.379955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:00.382369 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 413/826] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:00.390193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a18c29-dede-4dcb-b72e-51124c9f0bdb None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:00.391454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3e56e-fc96-4a54-a04f-04de4ed4e5fe None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:00.432307 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9e573-4e48-4ac5-89fd-f5286886d0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:00.433241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9e573-4e48-4ac5-89fd-f5286886d0be None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:00.433678 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9e573-4e48-4ac5-89fd-f5286886d0be None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:00.434132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9e573-4e48-4ac5-89fd-f5286886d0be None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:00.435000 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9e573-4e48-4ac5-89fd-f5286886d0be None admin] RBAC: Adding query filter params (type=network) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:00.437068 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9e573-4e48-4ac5-89fd-f5286886d0be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:00.438624 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9e573-4e48-4ac5-89fd-f5286886d0be None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:00.456297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 414/827] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jun 22 11:33:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:00.458912 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:00.468860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9e573-4e48-4ac5-89fd-f5286886d0be None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:00.578586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3e56e-fc96-4a54-a04f-04de4ed4e5fe None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:00.597828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:00.624243 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49931915-86db-4fec-b1a9-3ac08251d805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:00.625111 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49931915-86db-4fec-b1a9-3ac08251d805 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:00.625401 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49931915-86db-4fec-b1a9-3ac08251d805 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:00.625677 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49931915-86db-4fec-b1a9-3ac08251d805 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:00.626475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49931915-86db-4fec-b1a9-3ac08251d805 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:00.626815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49931915-86db-4fec-b1a9-3ac08251d805 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:00.629189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49931915-86db-4fec-b1a9-3ac08251d805 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:00.647072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 414/828] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:00.653209 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49931915-86db-4fec-b1a9-3ac08251d805 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:00.672879 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9e573-4e48-4ac5-89fd-f5286886d0be None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:00.742193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec0ad9-329a-4d2b-8dc6-1d2d5ea5c3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:00.743231 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec0ad9-329a-4d2b-8dc6-1d2d5ea5c3a1 None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:00.743775 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec0ad9-329a-4d2b-8dc6-1d2d5ea5c3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:00.744430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec0ad9-329a-4d2b-8dc6-1d2d5ea5c3a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:00.745401 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec0ad9-329a-4d2b-8dc6-1d2d5ea5c3a1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:00.746984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec0ad9-329a-4d2b-8dc6-1d2d5ea5c3a1 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:00.789979 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 415/829] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jun 22 11:33:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:33:00.818270 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:00.832544 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49931915-86db-4fec-b1a9-3ac08251d805 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:00.878380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a640ce-f611-4dcf-a5b3-816f496e764a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:00.879105 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a640ce-f611-4dcf-a5b3-816f496e764a None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:00.879414 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a640ce-f611-4dcf-a5b3-816f496e764a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:00.879696 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a640ce-f611-4dcf-a5b3-816f496e764a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:00.887806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a640ce-f611-4dcf-a5b3-816f496e764a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:00.889380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a640ce-f611-4dcf-a5b3-816f496e764a None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:00.906367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-d0a640ce-f611-4dcf-a5b3-816f496e764a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:00.906367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:00.906367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:00.906367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:00.906367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:00.906367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:00.906367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:00.906367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:00.906367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:00.906367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:00.906367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:00.906367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:00.906367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:33:00.906367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:33:00.906367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:33:00.907396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:33:00.907396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:00.907396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:00.907396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:00.907396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:00.907396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:00.907396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:00.907396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:00.907396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:00.907396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:00.907396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:00.907396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:00.907396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:00.907396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:00.907396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:00.907396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:00.907396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:00.908368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:33:00.908368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:33:00.908368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:33:00.908368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:33:00.908368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:33:00.908368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:33:00.908368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:00.908368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:00.912233 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 415/830] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:33:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:00.915893 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec0ad9-329a-4d2b-8dc6-1d2d5ea5c3a1 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:01.041944 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec0ad9-329a-4d2b-8dc6-1d2d5ea5c3a1 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:01.083739 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:01.089745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792a17f-0c13-46ab-a69d-a18dafece10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:01.090737 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792a17f-0c13-46ab-a69d-a18dafece10e None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:01.091800 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792a17f-0c13-46ab-a69d-a18dafece10e None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:01.092410 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792a17f-0c13-46ab-a69d-a18dafece10e None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:01.094115 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792a17f-0c13-46ab-a69d-a18dafece10e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:01.094816 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792a17f-0c13-46ab-a69d-a18dafece10e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:01.098648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792a17f-0c13-46ab-a69d-a18dafece10e None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:01.127826 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 416/831] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:33:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:01.130306 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a640ce-f611-4dcf-a5b3-816f496e764a None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:01.270115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a640ce-f611-4dcf-a5b3-816f496e764a None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:01.309045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f88ce-581b-40f2-a38e-d98884d47373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:01.309966 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f88ce-581b-40f2-a38e-d98884d47373 None admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:01.310280 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f88ce-581b-40f2-a38e-d98884d47373 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:01.310563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f88ce-581b-40f2-a38e-d98884d47373 None admin] PATH_INFO: `/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/dcd642e290e44e67b56eb3b8361d7af3/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:01.313110 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f88ce-581b-40f2-a38e-d98884d47373 None admin] RBAC: Authorizing `identity:create_grant(project_id=47d2bb094c9843c9a7726f1d672d2433, user_id=dcd642e290e44e67b56eb3b8361d7af3, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:01.316198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f88ce-581b-40f2-a38e-d98884d47373 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:01.336023 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 416/832] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:33:01 2022] PUT /identity/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/dcd642e290e44e67b56eb3b8361d7af3/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:33:01.937485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f3e47-d3ae-4088-8d72-e8b61e0be3a2 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:01.938057 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f3e47-d3ae-4088-8d72-e8b61e0be3a2 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:01.938057 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f3e47-d3ae-4088-8d72-e8b61e0be3a2 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:01.942056 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 417/833] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:01.950196 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feba7ec-ca80-4bc1-99c7-779466086fd3 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:01.950833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feba7ec-ca80-4bc1-99c7-779466086fd3 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:01.951319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feba7ec-ca80-4bc1-99c7-779466086fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:01.969001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-8feba7ec-ca80-4bc1-99c7-779466086fd3 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:01.970524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-8feba7ec-ca80-4bc1-99c7-779466086fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:01.973402 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:02.003814 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8ef0a-18d1-4a50-b7fe-a5d0c061cf02 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:02.004124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8ef0a-18d1-4a50-b7fe-a5d0c061cf02 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:02.004386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea8ef0a-18d1-4a50-b7fe-a5d0c061cf02 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:02.006006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 418/834] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:02.011164 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f113cb1-47ab-47e5-af7d-3774f934f584 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:02.011547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f113cb1-47ab-47e5-af7d-3774f934f584 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:02.011842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f113cb1-47ab-47e5-af7d-3774f934f584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:02.029281 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-3f113cb1-47ab-47e5-af7d-3774f934f584 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:02.030607 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-3f113cb1-47ab-47e5-af7d-3774f934f584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:02.046040 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 417/835] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:02.070633 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feba7ec-ca80-4bc1-99c7-779466086fd3 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:02.111338 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 419/836] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:33:02.132930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f113cb1-47ab-47e5-af7d-3774f934f584 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:02.199019 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feba7ec-ca80-4bc1-99c7-779466086fd3 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:02.235370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664216c-e13c-4889-a95f-248b9f1734c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:02.236568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664216c-e13c-4889-a95f-248b9f1734c5 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:02.237090 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664216c-e13c-4889-a95f-248b9f1734c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:02.237584 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664216c-e13c-4889-a95f-248b9f1734c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:02.246819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:02.250221 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f113cb1-47ab-47e5-af7d-3774f934f584 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:02.254325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664216c-e13c-4889-a95f-248b9f1734c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:33:02.263315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-b664216c-e13c-4889-a95f-248b9f1734c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:33:02.263315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:02.263315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:02.263315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:02.263315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:02.263315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:02.263315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:02.263315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:02.263315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:02.263315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:02.263315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:02.263315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:02.263315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:33:02.263315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:33:02.263315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:33:02.263315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:33:02.264308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:33:02.264308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:33:02.264308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:33:02.264308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:33:02.264308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:02.264308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:02.264308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:02.264308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:02.264308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:02.264308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:02.264308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:02.264308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:02.264308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:02.264308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:02.264308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:02.264308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:02.265436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:02.265436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:02.265436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:02.265436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:02.265436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:33:02.265436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:33:02.265436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:33:02.265436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:33:02.265436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:33:02.265436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:33:02.265436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:33:02.265436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:02.273430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 418/837] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:33:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:02.274696 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664216c-e13c-4889-a95f-248b9f1734c5 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:02.275974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:02.294413 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d69d1-758a-4b6e-bede-4b2f7ba0d2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:02.295297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d69d1-758a-4b6e-bede-4b2f7ba0d2a5 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:02.295642 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d69d1-758a-4b6e-bede-4b2f7ba0d2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:02.295924 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d69d1-758a-4b6e-bede-4b2f7ba0d2a5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:02.296597 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d69d1-758a-4b6e-bede-4b2f7ba0d2a5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:02.298169 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d69d1-758a-4b6e-bede-4b2f7ba0d2a5 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:02.312368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-821d69d1-758a-4b6e-bede-4b2f7ba0d2a5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 22 11:33:02.312368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:02.312368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:33:02.312368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:33:02.312368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:33:02.312368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:33:02.312368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:33:02.312368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:02.312368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 22 11:33:02.312368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:02.312368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:02.312368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:02.312368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:02.312368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:02.312368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:02.312368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:02.313367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:02.313367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:02.313367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:02.313367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:02.313367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:02.313367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:02.313367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:02.313367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:33:02.313367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:33:02.313367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:33:02.313367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:33:02.313367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:02.313367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:02.313367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:02.313367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:02.313367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:02.314206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:02.314206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:02.314206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:02.314206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:02.314206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:02.314206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:02.314206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:02.314206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:02.314206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:02.314206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:02.314206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:02.314206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:33:02.314206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:02.314206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 22 11:33:02.314206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:02.314206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 420/838] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Jun 22 11:33:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:02.319766 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d69d1-758a-4b6e-bede-4b2f7ba0d2a5 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:02.404443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664216c-e13c-4889-a95f-248b9f1734c5 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:02.441682 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26aa257-9a52-4c54-933f-0ea5eddc3b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:02.442368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26aa257-9a52-4c54-933f-0ea5eddc3b95 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:02.442651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26aa257-9a52-4c54-933f-0ea5eddc3b95 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:02.442927 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26aa257-9a52-4c54-933f-0ea5eddc3b95 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:02.443722 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26aa257-9a52-4c54-933f-0ea5eddc3b95 None admin] RBAC: Adding query filter params (name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:02.444032 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26aa257-9a52-4c54-933f-0ea5eddc3b95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:02.444136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d69d1-758a-4b6e-bede-4b2f7ba0d2a5 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:02.445419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26aa257-9a52-4c54-933f-0ea5eddc3b95 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:02.455029 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 419/839] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:33:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:02.456881 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb83292-e075-4f52-a725-3ae314c81e18 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:02.457223 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb83292-e075-4f52-a725-3ae314c81e18 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:02.457570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb83292-e075-4f52-a725-3ae314c81e18 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:02.459082 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 420/840] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:02.463680 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb83292-e075-4f52-a725-3ae314c81e18 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:02.484111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda27a33-27f3-4cd1-86fa-cb6c1531affd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:02.484687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda27a33-27f3-4cd1-86fa-cb6c1531affd None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:02.484971 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda27a33-27f3-4cd1-86fa-cb6c1531affd None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:02.485252 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda27a33-27f3-4cd1-86fa-cb6c1531affd None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:02.485955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda27a33-27f3-4cd1-86fa-cb6c1531affd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:02.486262 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda27a33-27f3-4cd1-86fa-cb6c1531affd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:02.488332 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda27a33-27f3-4cd1-86fa-cb6c1531affd None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:02.493072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 421/841] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jun 22 11:33:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:02.497450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d2f12-60e2-49a5-9dff-76a650234d03 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:02.497985 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d2f12-60e2-49a5-9dff-76a650234d03 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:02.498528 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d2f12-60e2-49a5-9dff-76a650234d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:02.521699 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-c40d2f12-60e2-49a5-9dff-76a650234d03 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:02.522905 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-c40d2f12-60e2-49a5-9dff-76a650234d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:02.538958 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:02.574834 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:02.585078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb83292-e075-4f52-a725-3ae314c81e18 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:02.604144 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 422/842] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:02 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:02.605023 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d2f12-60e2-49a5-9dff-76a650234d03 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:02.630703 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa379f-0e3e-469a-b79f-3c4a80f7bc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:02.631819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa379f-0e3e-469a-b79f-3c4a80f7bc9b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:02.632099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa379f-0e3e-469a-b79f-3c4a80f7bc9b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:02.632364 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa379f-0e3e-469a-b79f-3c4a80f7bc9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:02.642763 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa379f-0e3e-469a-b79f-3c4a80f7bc9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:02.644238 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa379f-0e3e-469a-b79f-3c4a80f7bc9b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:02.662103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-d0fa379f-0e3e-469a-b79f-3c4a80f7bc9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:02.662103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:02.662103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:02.662103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:02.662103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:02.662103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:02.662103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:02.662103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:02.662103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:02.662103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:02.662103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:02.662103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:02.662103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:33:02.662103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:33:02.662103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:33:02.663102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:33:02.663102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:02.663102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:02.663102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:02.663102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:02.663102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:02.663102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:02.663102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:02.663102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:02.663102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:02.663102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:02.663102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:02.663102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:02.663102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:02.663102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:02.663102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:02.663102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:02.664326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:33:02.664326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:33:02.664326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:02.664326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:02.664326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 421/843] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:02.664326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa379f-0e3e-469a-b79f-3c4a80f7bc9b None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:02.722918 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d2f12-60e2-49a5-9dff-76a650234d03 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:02.764065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e71ca7-d066-4652-aff1-f4eefa463680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:02.765346 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e71ca7-d066-4652-aff1-f4eefa463680 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:02.765630 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e71ca7-d066-4652-aff1-f4eefa463680 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:02.765907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e71ca7-d066-4652-aff1-f4eefa463680 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:02.766619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e71ca7-d066-4652-aff1-f4eefa463680 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:02.766923 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e71ca7-d066-4652-aff1-f4eefa463680 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:02.768904 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e71ca7-d066-4652-aff1-f4eefa463680 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:02.783612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 423/844] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jun 22 11:33:02 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:02.783612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e71ca7-d066-4652-aff1-f4eefa463680 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:02.793528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa379f-0e3e-469a-b79f-3c4a80f7bc9b None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:02.813333 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:02.834748 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:02.846789 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c595c20-d1ee-4532-994d-9ad8632d72d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:02.847728 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c595c20-d1ee-4532-994d-9ad8632d72d2 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:02.848015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c595c20-d1ee-4532-994d-9ad8632d72d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:02.848295 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c595c20-d1ee-4532-994d-9ad8632d72d2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:02.849013 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c595c20-d1ee-4532-994d-9ad8632d72d2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:02.850561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c595c20-d1ee-4532-994d-9ad8632d72d2 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:02.865144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-1c595c20-d1ee-4532-994d-9ad8632d72d2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 22 11:33:02.865144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:02.865144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:33:02.865144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:33:02.865144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:33:02.865144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:33:02.865144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:33:02.865144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:02.865144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 22 11:33:02.865144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:02.865144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:02.865144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:02.865144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:02.865144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:02.865144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:02.865144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:02.865144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:02.866391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:02.866391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:02.866391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:02.866391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:02.866391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:02.866391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:02.866391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:33:02.866391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:33:02.866391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:33:02.866391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:33:02.866391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:02.866391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:02.866391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:02.866391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:02.866391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:02.866391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:02.867252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:02.867252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:02.867252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:02.867252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:02.867252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:02.867252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:02.867252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:02.867252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:02.867252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:02.867252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:02.867252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:33:02.867252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:02.867252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 22 11:33:02.867252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:02.871759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 422/845] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:33:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:02.872848 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c595c20-d1ee-4532-994d-9ad8632d72d2 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:02.917778 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e71ca7-d066-4652-aff1-f4eefa463680 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:02.966492 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105e290-a3e8-4328-af8d-63afab0f0d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:02.967374 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105e290-a3e8-4328-af8d-63afab0f0d9d None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:02.967766 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105e290-a3e8-4328-af8d-63afab0f0d9d None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:02.968045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105e290-a3e8-4328-af8d-63afab0f0d9d None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:02.968776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105e290-a3e8-4328-af8d-63afab0f0d9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:02.969078 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105e290-a3e8-4328-af8d-63afab0f0d9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:02.970441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105e290-a3e8-4328-af8d-63afab0f0d9d None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:02.987691 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 424/846] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:02.993240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105e290-a3e8-4328-af8d-63afab0f0d9d None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:03.005480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c595c20-d1ee-4532-994d-9ad8632d72d2 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:03.044828 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5460af5-7b24-42f9-b4f7-80c72ea42f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:03.045845 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5460af5-7b24-42f9-b4f7-80c72ea42f22 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:03.046268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5460af5-7b24-42f9-b4f7-80c72ea42f22 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:03.046684 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5460af5-7b24-42f9-b4f7-80c72ea42f22 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:03.047646 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5460af5-7b24-42f9-b4f7-80c72ea42f22 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:03.048185 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5460af5-7b24-42f9-b4f7-80c72ea42f22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:03.049941 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5460af5-7b24-42f9-b4f7-80c72ea42f22 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:03.055610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 423/847] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:33:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:03.061426 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5460af5-7b24-42f9-b4f7-80c72ea42f22 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:03.116769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:03.118663 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105e290-a3e8-4328-af8d-63afab0f0d9d None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:03.120746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:03.158126 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1093a-7f17-452a-bf89-da52910b1119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:03.159923 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1093a-7f17-452a-bf89-da52910b1119 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:03.160428 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1093a-7f17-452a-bf89-da52910b1119 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:03.160898 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1093a-7f17-452a-bf89-da52910b1119 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:03.170519 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f1093a-7f17-452a-bf89-da52910b1119 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:03.172684 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f1093a-7f17-452a-bf89-da52910b1119 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:03.188192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-79f1093a-7f17-452a-bf89-da52910b1119 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 22 11:33:03.188192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:03.188192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:03.188192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:03.188192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:03.188192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:03.188192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:03.188192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:03.188192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:03.188192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:03.188192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:03.188192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:03.188192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:33:03.188192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:33:03.188192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:33:03.188192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:33:03.189305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:03.189305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:03.189305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:33:03.189305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:03.189305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:33:03.189305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:03.189305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:03.189305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:03.189305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:03.189305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:03.189305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:03.189305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:03.189305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:03.189305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:03.189305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:03.189305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:03.189305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:03.189305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:03.190310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:03.190310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:03.190310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:33:03.190310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:33:03.190310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:33:03.190310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:33:03.190310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:33:03.190310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:33:03.190310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 22 11:33:03.190310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:03.191404 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5460af5-7b24-42f9-b4f7-80c72ea42f22 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:03.193302 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 425/848] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jun 22 11:33:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:03.199402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1093a-7f17-452a-bf89-da52910b1119 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:03.247960 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b028f0e-b4a4-4dfa-b993-045ac344b469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:03.248607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b028f0e-b4a4-4dfa-b993-045ac344b469 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:03.248838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b028f0e-b4a4-4dfa-b993-045ac344b469 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:03.249116 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b028f0e-b4a4-4dfa-b993-045ac344b469 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:03.249831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b028f0e-b4a4-4dfa-b993-045ac344b469 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:03.250142 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b028f0e-b4a4-4dfa-b993-045ac344b469 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:03.252097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b028f0e-b4a4-4dfa-b993-045ac344b469 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:03.263456 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 424/849] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:33:03 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:03.336322 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1093a-7f17-452a-bf89-da52910b1119 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:03.378614 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dda47c-7ef9-4d81-8053-0c21c37d78fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:03.379641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dda47c-7ef9-4d81-8053-0c21c37d78fb None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:03.380071 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dda47c-7ef9-4d81-8053-0c21c37d78fb None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:03.380493 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dda47c-7ef9-4d81-8053-0c21c37d78fb None admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:03.381539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dda47c-7ef9-4d81-8053-0c21c37d78fb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:03.382027 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dda47c-7ef9-4d81-8053-0c21c37d78fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:03.383787 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dda47c-7ef9-4d81-8053-0c21c37d78fb None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:03.405501 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 426/850] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jun 22 11:33:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:03.411946 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b028f0e-b4a4-4dfa-b993-045ac344b469 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:03.523152 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b028f0e-b4a4-4dfa-b993-045ac344b469 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:03.525965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:03.542312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609ae99-3e9b-424b-a11e-c48653b8a352 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:03.542802 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609ae99-3e9b-424b-a11e-c48653b8a352 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:03.543455 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609ae99-3e9b-424b-a11e-c48653b8a352 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:03.545245 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 427/851] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:33:03.552542 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbe2b5-359b-4736-81f7-5cd3cb150e13 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:03.552746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbe2b5-359b-4736-81f7-5cd3cb150e13 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:03.552959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbe2b5-359b-4736-81f7-5cd3cb150e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:03.564641 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2579c27-1d57-4f7e-9678-9bdbf54b4b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:03.565872 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2579c27-1d57-4f7e-9678-9bdbf54b4b72 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:03.566401 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2579c27-1d57-4f7e-9678-9bdbf54b4b72 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:03.566893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2579c27-1d57-4f7e-9678-9bdbf54b4b72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:03.573320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-8cdbe2b5-359b-4736-81f7-5cd3cb150e13 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:03.574522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-8cdbe2b5-359b-4736-81f7-5cd3cb150e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:03.594369 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:03.650809 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2579c27-1d57-4f7e-9678-9bdbf54b4b72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:03.652799 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2579c27-1d57-4f7e-9678-9bdbf54b4b72 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:03.779169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 425/852] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:33:03 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 22 11:33:03.785326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2579c27-1d57-4f7e-9678-9bdbf54b4b72 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:03.789059 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:03.803164 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 428/853] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:03 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:33:03.826112 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbe2b5-359b-4736-81f7-5cd3cb150e13 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:03.917388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2579c27-1d57-4f7e-9678-9bdbf54b4b72 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:03.946599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbe2b5-359b-4736-81f7-5cd3cb150e13 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:03.971386 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee50d9-2cb8-4c48-8d86-b14893848d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:03.973043 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee50d9-2cb8-4c48-8d86-b14893848d37 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:03.973413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee50d9-2cb8-4c48-8d86-b14893848d37 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:03.973700 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee50d9-2cb8-4c48-8d86-b14893848d37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:03.983365 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497564ef-b8d7-45f5-8100-ec5d0014311a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:03.984292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497564ef-b8d7-45f5-8100-ec5d0014311a None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:03.984642 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497564ef-b8d7-45f5-8100-ec5d0014311a None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:03.984985 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497564ef-b8d7-45f5-8100-ec5d0014311a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:03.987678 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee50d9-2cb8-4c48-8d86-b14893848d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:03.989645 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee50d9-2cb8-4c48-8d86-b14893848d37 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:03.998112 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497564ef-b8d7-45f5-8100-ec5d0014311a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:33:04.006400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-497564ef-b8d7-45f5-8100-ec5d0014311a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:33:04.006400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:04.006400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:04.006400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:04.006400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:04.006400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:04.006400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:04.006400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:04.006400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:04.006400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:04.006400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:04.006400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:04.006400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:33:04.006400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:33:04.006400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:33:04.006400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:33:04.007386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:33:04.007386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:33:04.007386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:33:04.007386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:33:04.007386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:04.007386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:04.007386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:04.007386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:04.007386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:04.007386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:04.007386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:04.007386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:04.007386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:04.007386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:04.007386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:04.007386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:04.008247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:04.008247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:04.008247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:04.008247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:04.008247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:33:04.008247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:33:04.008247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:33:04.008247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:33:04.008247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:33:04.008247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:33:04.008247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:33:04.008247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:04.008247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 429/854] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:33:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:04.011473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-03ee50d9-2cb8-4c48-8d86-b14893848d37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:04.011473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:04.011473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:04.011473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:04.011473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:04.011473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:04.011473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:04.011473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:04.011473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:04.011473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:04.011473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:04.011473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:04.011473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:33:04.011473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:33:04.011473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:33:04.013385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:33:04.013385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:04.013385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:04.013385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:04.013385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:04.013385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:04.013385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:04.013385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:04.013385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:04.013385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:04.013385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:04.013385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:04.013385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:04.013385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:04.013385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:04.013385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:04.013385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:04.015705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:33:04.015705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:33:04.015705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:04.015705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:04.018525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497564ef-b8d7-45f5-8100-ec5d0014311a None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:04.018949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 426/855] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:04.020457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee50d9-2cb8-4c48-8d86-b14893848d37 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:04.032503 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:04.099223 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:04.132307 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497564ef-b8d7-45f5-8100-ec5d0014311a None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:04.157984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee50d9-2cb8-4c48-8d86-b14893848d37 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:04.167332 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44558c8-50a7-4780-a233-fe67d23bc8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:04.168028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44558c8-50a7-4780-a233-fe67d23bc8e9 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:04.168309 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44558c8-50a7-4780-a233-fe67d23bc8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:04.168585 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44558c8-50a7-4780-a233-fe67d23bc8e9 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:04.169329 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44558c8-50a7-4780-a233-fe67d23bc8e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:04.169630 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44558c8-50a7-4780-a233-fe67d23bc8e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:04.171002 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44558c8-50a7-4780-a233-fe67d23bc8e9 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:04.184738 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 430/856] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:33:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:04.186008 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44558c8-50a7-4780-a233-fe67d23bc8e9 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:04.200663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098aa36-3d16-44cd-9618-136a18c98f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:04.201973 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098aa36-3d16-44cd-9618-136a18c98f46 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:04.202753 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098aa36-3d16-44cd-9618-136a18c98f46 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:04.203260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098aa36-3d16-44cd-9618-136a18c98f46 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:04.204424 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098aa36-3d16-44cd-9618-136a18c98f46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:04.205227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098aa36-3d16-44cd-9618-136a18c98f46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:04.206850 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098aa36-3d16-44cd-9618-136a18c98f46 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:04.223917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 427/857] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:04.231871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098aa36-3d16-44cd-9618-136a18c98f46 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:04.263275 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:04.297190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44558c8-50a7-4780-a233-fe67d23bc8e9 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:04.337308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcf919-274f-4443-8c56-cb909d2ef366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:04.338486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcf919-274f-4443-8c56-cb909d2ef366 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:04.338840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcf919-274f-4443-8c56-cb909d2ef366 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:04.339206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcf919-274f-4443-8c56-cb909d2ef366 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:04.354853 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bcf919-274f-4443-8c56-cb909d2ef366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:04.356847 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bcf919-274f-4443-8c56-cb909d2ef366 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:04.363323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098aa36-3d16-44cd-9618-136a18c98f46 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:04.382184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-e3bcf919-274f-4443-8c56-cb909d2ef366 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:04.382184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:04.382184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:04.382184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:04.382184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:04.382184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:04.382184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:04.382184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:04.382184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:04.382184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:04.382184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:04.382184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:04.382184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:33:04.382184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:33:04.382184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:33:04.383167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:33:04.383167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:04.383167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:04.383167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:04.383167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:04.383167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:04.383167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:04.383167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:04.383167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:04.383167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:04.383167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:04.383167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:04.383167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:04.383167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:04.383167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:04.383167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:04.383167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:04.384096 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:33:04.384096 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:33:04.384096 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:04.384096 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:04.386161 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:04.392125 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 431/858] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:04.394414 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcf919-274f-4443-8c56-cb909d2ef366 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:04.415086 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf343a9-657a-4a26-ae56-5b51389653f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:04.416822 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf343a9-657a-4a26-ae56-5b51389653f2 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:04.417254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf343a9-657a-4a26-ae56-5b51389653f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:04.417673 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf343a9-657a-4a26-ae56-5b51389653f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:04.427183 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf343a9-657a-4a26-ae56-5b51389653f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:04.429830 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf343a9-657a-4a26-ae56-5b51389653f2 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:04.449205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-faf343a9-657a-4a26-ae56-5b51389653f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:04.449205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:04.449205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:04.449205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:04.449205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:04.449205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:04.449205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:04.449205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:04.449205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:04.449205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:04.449205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:04.449205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:04.449205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:33:04.449205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:33:04.449205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:33:04.450338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:33:04.450338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:04.450338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:04.450338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:04.450338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:04.450338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:04.450338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:04.450338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:04.450338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:04.450338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:04.450338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:04.450338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:04.450338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:04.450338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:04.450338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:04.450338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:04.450338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:04.451458 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:33:04.451458 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:33:04.451458 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:33:04.451458 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:33:04.451458 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:33:04.451458 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:33:04.451458 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:04.451458 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:04.454173 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 428/859] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:33:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:04.460174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf343a9-657a-4a26-ae56-5b51389653f2 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:04.525728 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcf919-274f-4443-8c56-cb909d2ef366 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:04.553676 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:04.565912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808bf826-a0f9-4237-8028-cb39edb01c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:04.566703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808bf826-a0f9-4237-8028-cb39edb01c60 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:04.567104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808bf826-a0f9-4237-8028-cb39edb01c60 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:04.567511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808bf826-a0f9-4237-8028-cb39edb01c60 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:04.568341 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808bf826-a0f9-4237-8028-cb39edb01c60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:04.568728 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808bf826-a0f9-4237-8028-cb39edb01c60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:04.570191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808bf826-a0f9-4237-8028-cb39edb01c60 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:04.582702 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf343a9-657a-4a26-ae56-5b51389653f2 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:04.588642 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 432/860] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:04.590056 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808bf826-a0f9-4237-8028-cb39edb01c60 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:04.627316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30688e36-fb06-4787-8494-f39cbd58b3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:04.628313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30688e36-fb06-4787-8494-f39cbd58b3de None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:04.628744 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30688e36-fb06-4787-8494-f39cbd58b3de None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:04.629263 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30688e36-fb06-4787-8494-f39cbd58b3de None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:04.630986 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30688e36-fb06-4787-8494-f39cbd58b3de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:04.631439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30688e36-fb06-4787-8494-f39cbd58b3de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:04.632970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30688e36-fb06-4787-8494-f39cbd58b3de None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:04.651429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 429/861] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:33:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:04.656300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30688e36-fb06-4787-8494-f39cbd58b3de None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:04.702738 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:04.722546 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808bf826-a0f9-4237-8028-cb39edb01c60 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:04.773864 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a187c8-e23a-49ba-a6e1-055550000416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:04.774986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a187c8-e23a-49ba-a6e1-055550000416 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:04.775439 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a187c8-e23a-49ba-a6e1-055550000416 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:04.775883 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a187c8-e23a-49ba-a6e1-055550000416 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:04.787796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a187c8-e23a-49ba-a6e1-055550000416 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:04.789832 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a187c8-e23a-49ba-a6e1-055550000416 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:04.790439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30688e36-fb06-4787-8494-f39cbd58b3de None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:04.813790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-04a187c8-e23a-49ba-a6e1-055550000416 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 22 11:33:04.813790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:04.813790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:04.813790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:04.813790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:04.813790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:04.813790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:04.813790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:04.813790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:04.813790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:04.813790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:04.813790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:04.813790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:33:04.813790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:33:04.813790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:33:04.813790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:33:04.815557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:04.815557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:04.815557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:33:04.815557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:04.815557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:33:04.815557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:04.815557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:04.815557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:04.815557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:04.815557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:04.815557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:04.815557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:04.815557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:04.815557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:04.815557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:04.815557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:04.815557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:04.815557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:04.816510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:04.816510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:04.816510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:33:04.816510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:33:04.816510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:33:04.816510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:33:04.816510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:33:04.816510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:33:04.816510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 22 11:33:04.816510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:04.819805 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 433/862] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:33:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:04.826116 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a187c8-e23a-49ba-a6e1-055550000416 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:04.841635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b67c2-c850-47be-b40a-12d3614319ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:04.842752 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b67c2-c850-47be-b40a-12d3614319ae None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:04.843292 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b67c2-c850-47be-b40a-12d3614319ae None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:04.843828 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b67c2-c850-47be-b40a-12d3614319ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:04.844901 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b67c2-c850-47be-b40a-12d3614319ae None admin] RBAC: Adding query filter params (role.id=64c2d69f15ca458c8267c1acfd42d9fc, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=955c44ffe1bd426e900c7405df44d1cf) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:04.845530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b67c2-c850-47be-b40a-12d3614319ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:04.847375 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b67c2-c850-47be-b40a-12d3614319ae None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:04.854877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 430/863] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:33:04 2022] GET /identity/v3/role_assignments?user.id=955c44ffe1bd426e900c7405df44d1cf&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=64c2d69f15ca458c8267c1acfd42d9fc => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:04.892675 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:04.951407 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7ba03-b61f-4120-ab6f-69a97e1eac0f None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:04.951733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7ba03-b61f-4120-ab6f-69a97e1eac0f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:04.951978 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7ba03-b61f-4120-ab6f-69a97e1eac0f None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:04.953987 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 431/864] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:33:04.959758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a187c8-e23a-49ba-a6e1-055550000416 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:04.959972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcde3e7-cf9b-4a8e-97d7-4b52aefea145 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:04.960219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcde3e7-cf9b-4a8e-97d7-4b52aefea145 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:04.960502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcde3e7-cf9b-4a8e-97d7-4b52aefea145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:04.980086 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-2bcde3e7-cf9b-4a8e-97d7-4b52aefea145 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:04.981300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2bcde3e7-cf9b-4a8e-97d7-4b52aefea145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:05.001905 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4923359-57f0-4a2d-becf-0a3f26acf537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:05.002613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4923359-57f0-4a2d-becf-0a3f26acf537 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:05.002902 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4923359-57f0-4a2d-becf-0a3f26acf537 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:05.003182 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4923359-57f0-4a2d-becf-0a3f26acf537 None admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:05.004071 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4923359-57f0-4a2d-becf-0a3f26acf537 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:05.004387 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4923359-57f0-4a2d-becf-0a3f26acf537 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:05.005851 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4923359-57f0-4a2d-becf-0a3f26acf537 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:05.030940 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 434/865] 10.222.0.36 () {56 vars in 1135 bytes} [Wed Jun 22 11:33:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:05.032818 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4923359-57f0-4a2d-becf-0a3f26acf537 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:05.069757 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 432/866] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:05.093781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcde3e7-cf9b-4a8e-97d7-4b52aefea145 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:05.126926 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:05.154713 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4923359-57f0-4a2d-becf-0a3f26acf537 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:05.156946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:05.198509 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a84f8-f8c8-47d8-bf68-8ead06b93b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:05.199377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a84f8-f8c8-47d8-bf68-8ead06b93b5f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:05.199992 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a84f8-f8c8-47d8-bf68-8ead06b93b5f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:05.200285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a84f8-f8c8-47d8-bf68-8ead06b93b5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:05.215623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcde3e7-cf9b-4a8e-97d7-4b52aefea145 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:05.269585 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42529d8c-b619-4346-84ef-33af797ef498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:05.270672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42529d8c-b619-4346-84ef-33af797ef498 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:05.271162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42529d8c-b619-4346-84ef-33af797ef498 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:05.271659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42529d8c-b619-4346-84ef-33af797ef498 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:05.272592 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42529d8c-b619-4346-84ef-33af797ef498 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:05.274166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42529d8c-b619-4346-84ef-33af797ef498 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:05.286902 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a84f8-f8c8-47d8-bf68-8ead06b93b5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:05.288483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a84f8-f8c8-47d8-bf68-8ead06b93b5f None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:05.292940 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 433/867] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jun 22 11:33:05 2022] POST /identity/v3/services => generated 269 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:33:05.352912 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4ff1f-2b1d-4c36-a71a-54dcff948960 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:05.353535 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4ff1f-2b1d-4c36-a71a-54dcff948960 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:05.354238 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4ff1f-2b1d-4c36-a71a-54dcff948960 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:05.356762 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 434/868] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:33:05.364802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522b830-cd67-436f-bfac-7409d1eefceb None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:05.365438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522b830-cd67-436f-bfac-7409d1eefceb None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:05.365438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2522b830-cd67-436f-bfac-7409d1eefceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:05.384063 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-2522b830-cd67-436f-bfac-7409d1eefceb None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:05.385317 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2522b830-cd67-436f-bfac-7409d1eefceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:05.427818 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 435/869] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:33:05 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 22 11:33:05.433249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a84f8-f8c8-47d8-bf68-8ead06b93b5f None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:05.435448 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:05.477698 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 435/870] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:05 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:05.500967 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522b830-cd67-436f-bfac-7409d1eefceb None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:05.527086 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:05.567840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a84f8-f8c8-47d8-bf68-8ead06b93b5f None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:05.615785 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e20f1-3c2f-41d4-8329-71643e868e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:05.616502 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e20f1-3c2f-41d4-8329-71643e868e46 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:05.616785 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e20f1-3c2f-41d4-8329-71643e868e46 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:05.617112 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e20f1-3c2f-41d4-8329-71643e868e46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:05.632361 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e20f1-3c2f-41d4-8329-71643e868e46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:05.633823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e20f1-3c2f-41d4-8329-71643e868e46 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:05.634477 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2522b830-cd67-436f-bfac-7409d1eefceb None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:05.657458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-320e20f1-3c2f-41d4-8329-71643e868e46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:05.657458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:05.657458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:05.657458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:05.657458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:05.657458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:05.657458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:05.657458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:05.657458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:05.657458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:05.657458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:05.657458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:05.657458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:33:05.657458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:33:05.657458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:33:05.658412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:33:05.658412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:05.658412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:05.658412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:05.658412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:05.658412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:05.658412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:05.658412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:05.658412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:05.658412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:05.658412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:05.658412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:05.658412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:05.658412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:05.658412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:05.658412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:05.658412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:05.659365 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:33:05.659365 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:33:05.659365 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:05.659365 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:05.663513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 436/871] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:05.664753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e20f1-3c2f-41d4-8329-71643e868e46 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:05.678161 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1c292-f27f-43cf-9bb6-a529a417564c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:05.679176 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1c292-f27f-43cf-9bb6-a529a417564c None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:05.679680 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1c292-f27f-43cf-9bb6-a529a417564c None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:05.680152 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1c292-f27f-43cf-9bb6-a529a417564c None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:05.681096 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1c292-f27f-43cf-9bb6-a529a417564c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:05.682660 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1c292-f27f-43cf-9bb6-a529a417564c None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:05.710808 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 436/872] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jun 22 11:33:05 2022] POST /identity/v3/services => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:33:05.737728 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:05.795755 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e20f1-3c2f-41d4-8329-71643e868e46 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:05.844924 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0fee8-fc6e-49e8-a2bb-56538d24716b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:05.845599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0fee8-fc6e-49e8-a2bb-56538d24716b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:05.845899 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0fee8-fc6e-49e8-a2bb-56538d24716b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:05.846792 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0fee8-fc6e-49e8-a2bb-56538d24716b None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:05.847571 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0fee8-fc6e-49e8-a2bb-56538d24716b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:05.847918 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0fee8-fc6e-49e8-a2bb-56538d24716b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:05.849368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0fee8-fc6e-49e8-a2bb-56538d24716b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:05.867424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 437/873] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:05.868817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1c292-f27f-43cf-9bb6-a529a417564c None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:05.972931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:06.004331 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1c292-f27f-43cf-9bb6-a529a417564c None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:06.042213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35a13c-0ebe-468c-986e-ad5134dee1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:06.043513 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35a13c-0ebe-468c-986e-ad5134dee1b0 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:06.044010 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35a13c-0ebe-468c-986e-ad5134dee1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:06.044487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35a13c-0ebe-468c-986e-ad5134dee1b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:06.052685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35a13c-0ebe-468c-986e-ad5134dee1b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:06.054501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35a13c-0ebe-468c-986e-ad5134dee1b0 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:06.070212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-8e35a13c-0ebe-468c-986e-ad5134dee1b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:06.070212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:06.070212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:06.070212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:06.070212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:06.070212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:06.070212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:06.070212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:06.070212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:06.070212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:06.070212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:06.070212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:06.070212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:33:06.070212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:33:06.070212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:33:06.071167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:33:06.071167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:06.071167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:06.071167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:06.071167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:06.071167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:06.071167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:06.071167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:06.071167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:06.071167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:06.071167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:06.071167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:06.071167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:06.071167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:06.071167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:06.071167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:06.071167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:06.072501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:33:06.072501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:33:06.072501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:33:06.072501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:33:06.072501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:33:06.072501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:33:06.072501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:06.072501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:06.074844 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 437/874] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:33:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:06.081088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0fee8-fc6e-49e8-a2bb-56538d24716b None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:06.201649 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0fee8-fc6e-49e8-a2bb-56538d24716b None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:06.215171 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:06.241987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c12a1c-9326-4404-945d-5b5d05886437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:06.243045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c12a1c-9326-4404-945d-5b5d05886437 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:06.243515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c12a1c-9326-4404-945d-5b5d05886437 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:06.243943 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c12a1c-9326-4404-945d-5b5d05886437 None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:06.244901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c12a1c-9326-4404-945d-5b5d05886437 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:06.245363 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c12a1c-9326-4404-945d-5b5d05886437 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:06.247253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c12a1c-9326-4404-945d-5b5d05886437 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:06.262724 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 438/875] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:33:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:06.269020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35a13c-0ebe-468c-986e-ad5134dee1b0 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:06.394171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35a13c-0ebe-468c-986e-ad5134dee1b0 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:06.433922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fca82e-ca23-48fa-be42-d9f18fbf1bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:06.434883 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fca82e-ca23-48fa-be42-d9f18fbf1bbc None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:06.435320 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fca82e-ca23-48fa-be42-d9f18fbf1bbc None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:06.435769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fca82e-ca23-48fa-be42-d9f18fbf1bbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:06.436783 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fca82e-ca23-48fa-be42-d9f18fbf1bbc None admin] RBAC: Adding query filter params (role.id=96c584428d924fdba9671949f760c406, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=dcd642e290e44e67b56eb3b8361d7af3) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:06.437248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fca82e-ca23-48fa-be42-d9f18fbf1bbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:06.438869 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fca82e-ca23-48fa-be42-d9f18fbf1bbc None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:06.442757 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:06.446713 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 438/876] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:33:06 2022] GET /identity/v3/role_assignments?user.id=dcd642e290e44e67b56eb3b8361d7af3&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=96c584428d924fdba9671949f760c406 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:07.172369 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf059c3-8a19-4322-8fed-52b817b1ac14 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:07.172943 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf059c3-8a19-4322-8fed-52b817b1ac14 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:07.172943 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf059c3-8a19-4322-8fed-52b817b1ac14 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:07.174588 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 439/877] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:07.179986 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d6cdf-5d62-44b0-9c4e-a2d516c0f83d None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:07.180596 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d6cdf-5d62-44b0-9c4e-a2d516c0f83d None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:07.181086 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d6cdf-5d62-44b0-9c4e-a2d516c0f83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:07.199234 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-4f4d6cdf-5d62-44b0-9c4e-a2d516c0f83d None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:07.200781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-4f4d6cdf-5d62-44b0-9c4e-a2d516c0f83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:07.287341 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 439/878] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:07.310775 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf059c3-8a19-4322-8fed-52b817b1ac14 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:07.414389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf059c3-8a19-4322-8fed-52b817b1ac14 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:07.449505 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2cd1e-64c3-4690-b3b0-f1a1b644bd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:07.450677 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2cd1e-64c3-4690-b3b0-f1a1b644bd21 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:07.451174 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2cd1e-64c3-4690-b3b0-f1a1b644bd21 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:07.451712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2cd1e-64c3-4690-b3b0-f1a1b644bd21 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:07.464233 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c2cd1e-64c3-4690-b3b0-f1a1b644bd21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:33:07.473522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-53c2cd1e-64c3-4690-b3b0-f1a1b644bd21 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 22 11:33:07.473522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:07.473522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:07.473522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:07.473522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:07.473522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:07.473522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:07.473522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:07.473522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:07.473522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:07.473522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:07.473522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:07.473522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:33:07.473522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:33:07.473522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:33:07.473522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:33:07.474596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:33:07.474596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:33:07.474596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:33:07.474596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:33:07.474596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:07.474596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:07.474596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:07.474596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:07.474596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:07.474596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:07.474596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:07.474596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:07.474596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:07.474596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:07.474596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:07.474596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:07.475550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:07.475550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:07.475550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:07.475550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:07.475550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:33:07.475550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:33:07.475550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:33:07.475550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:33:07.475550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:33:07.475550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:33:07.475550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 22 11:33:07.475550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:07.478466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 440/879] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jun 22 11:33:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:07.484482 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d6cdf-5d62-44b0-9c4e-a2d516c0f83d None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:07.524479 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b860d-e62a-4519-abe5-e22cc4dc8dd2 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:07.525137 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b860d-e62a-4519-abe5-e22cc4dc8dd2 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:07.525632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b860d-e62a-4519-abe5-e22cc4dc8dd2 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:07.527499 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 441/880] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:07.533321 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc9c51-962c-4595-bf5a-1413f1cf8670 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:07.534478 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc9c51-962c-4595-bf5a-1413f1cf8670 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:07.534998 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc9c51-962c-4595-bf5a-1413f1cf8670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:07.552912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-effc9c51-962c-4595-bf5a-1413f1cf8670 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:07.554442 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-effc9c51-962c-4595-bf5a-1413f1cf8670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:07.579572 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:07.603892 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d6cdf-5d62-44b0-9c4e-a2d516c0f83d None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:07.639613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 442/881] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:07 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:33:07.643338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:07.650268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3448f70-6398-4d49-90da-3fa089839cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:07.650943 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3448f70-6398-4d49-90da-3fa089839cc6 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:07.651227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3448f70-6398-4d49-90da-3fa089839cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:07.651552 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3448f70-6398-4d49-90da-3fa089839cc6 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:07.652305 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3448f70-6398-4d49-90da-3fa089839cc6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:07.652613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3448f70-6398-4d49-90da-3fa089839cc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:07.653970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3448f70-6398-4d49-90da-3fa089839cc6 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:07.662737 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 440/882] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jun 22 11:33:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:07.663465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc9c51-962c-4595-bf5a-1413f1cf8670 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:07.672269 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3448f70-6398-4d49-90da-3fa089839cc6 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:07.806158 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3448f70-6398-4d49-90da-3fa089839cc6 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:07.821573 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc9c51-962c-4595-bf5a-1413f1cf8670 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:07.854007 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d730cc-933f-4d3e-b4d5-f57f6ee2291b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:07.855121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d730cc-933f-4d3e-b4d5-f57f6ee2291b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:07.855448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d730cc-933f-4d3e-b4d5-f57f6ee2291b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:07.856219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d730cc-933f-4d3e-b4d5-f57f6ee2291b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:07.861304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c29b1-c567-4790-894c-bff97a81846c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:07.862571 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c29b1-c567-4790-894c-bff97a81846c None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:07.863085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c29b1-c567-4790-894c-bff97a81846c None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:07.863631 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c29b1-c567-4790-894c-bff97a81846c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:07.864623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c29b1-c567-4790-894c-bff97a81846c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:07.866229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c29b1-c567-4790-894c-bff97a81846c None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:07.869955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d730cc-933f-4d3e-b4d5-f57f6ee2291b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:07.871745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d730cc-933f-4d3e-b4d5-f57f6ee2291b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:07.881457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-e09c29b1-c567-4790-894c-bff97a81846c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 22 11:33:07.881457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:07.881457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:33:07.881457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:33:07.881457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:33:07.881457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:33:07.881457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:33:07.881457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:07.881457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 22 11:33:07.881457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:07.881457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:07.881457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:07.881457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:07.881457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:07.881457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:07.881457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:07.882471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:07.882471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:07.882471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:07.882471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:07.882471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:07.882471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:07.882471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:07.882471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:33:07.882471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:33:07.882471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:33:07.882471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:33:07.882471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:07.882471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:07.882471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:07.882471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:07.882471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:07.883314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:07.883314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:07.883314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:07.883314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:07.883314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:07.883314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:07.883314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:07.883314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:07.883314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:07.883314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:07.883314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:07.883314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:33:07.883314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:07.883314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 22 11:33:07.883314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:07.886353 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 443/883] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Jun 22 11:33:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:07.890524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-32d730cc-933f-4d3e-b4d5-f57f6ee2291b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:07.890524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:07.890524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:07.890524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:07.890524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:07.890524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:07.890524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:07.890524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:07.890524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:07.890524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:07.890524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:07.890524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:07.890524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:33:07.890524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:33:07.890524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:33:07.891454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:33:07.891454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:07.891454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:07.891454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:07.891454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:07.891454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:07.891454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:07.891454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:07.891454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:07.891454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:07.891454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:07.891454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:07.891454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:07.891454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:07.891454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:07.891454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:07.891454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:07.892379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:33:07.892379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:33:07.892379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:07.892379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:07.893792 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c29b1-c567-4790-894c-bff97a81846c None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:07.895053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 441/884] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:07.901177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d730cc-933f-4d3e-b4d5-f57f6ee2291b None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:07.932106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:07.972697 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:08.039196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c29b1-c567-4790-894c-bff97a81846c None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:08.039460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d730cc-933f-4d3e-b4d5-f57f6ee2291b None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:08.077432 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d780d9-ec53-4bc4-ac8b-69caab577c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:08.078108 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d780d9-ec53-4bc4-ac8b-69caab577c75 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:08.078493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d780d9-ec53-4bc4-ac8b-69caab577c75 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:08.078897 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d780d9-ec53-4bc4-ac8b-69caab577c75 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:08.079572 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d780d9-ec53-4bc4-ac8b-69caab577c75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:08.079847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d780d9-ec53-4bc4-ac8b-69caab577c75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:08.079962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4a05f-c3ff-4c01-bc31-2d77e50ba1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:08.081165 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4a05f-c3ff-4c01-bc31-2d77e50ba1ef None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:08.081389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4a05f-c3ff-4c01-bc31-2d77e50ba1ef None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:08.081663 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4a05f-c3ff-4c01-bc31-2d77e50ba1ef None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:08.081767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d780d9-ec53-4bc4-ac8b-69caab577c75 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:08.082376 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4a05f-c3ff-4c01-bc31-2d77e50ba1ef None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:08.082680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4a05f-c3ff-4c01-bc31-2d77e50ba1ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:08.084422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4a05f-c3ff-4c01-bc31-2d77e50ba1ef None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:08.094052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 444/885] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jun 22 11:33:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:08.094052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189dbe9-befd-4b31-bc87-c2e2dcdc1017 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:08.094052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189dbe9-befd-4b31-bc87-c2e2dcdc1017 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:08.094052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189dbe9-befd-4b31-bc87-c2e2dcdc1017 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:08.094052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 445/886] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:08.096646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189dbe9-befd-4b31-bc87-c2e2dcdc1017 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:08.099198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 442/887] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:08.100534 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8c841-df28-49d3-88f2-0395fd826e86 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:08.100832 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8c841-df28-49d3-88f2-0395fd826e86 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:08.101113 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8c841-df28-49d3-88f2-0395fd826e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:08.118097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-a3e8c841-df28-49d3-88f2-0395fd826e86 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:08.119299 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a3e8c841-df28-49d3-88f2-0395fd826e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:08.188930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 443/888] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:08.189779 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8c841-df28-49d3-88f2-0395fd826e86 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:08.220602 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:08.235227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189dbe9-befd-4b31-bc87-c2e2dcdc1017 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:08.239908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:08.271107 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf6813-e0d3-45e3-be17-1118c4cc3d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:08.271800 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf6813-e0d3-45e3-be17-1118c4cc3d19 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:08.272094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf6813-e0d3-45e3-be17-1118c4cc3d19 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:08.272375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf6813-e0d3-45e3-be17-1118c4cc3d19 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:08.273084 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf6813-e0d3-45e3-be17-1118c4cc3d19 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:08.273392 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf6813-e0d3-45e3-be17-1118c4cc3d19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:08.274774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf6813-e0d3-45e3-be17-1118c4cc3d19 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:08.279986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 446/889] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jun 22 11:33:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:08.280769 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf6813-e0d3-45e3-be17-1118c4cc3d19 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:08.319802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8c841-df28-49d3-88f2-0395fd826e86 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:08.358799 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69352b5a-b974-4c0a-a5aa-a841ddde96e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:08.359566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69352b5a-b974-4c0a-a5aa-a841ddde96e1 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:08.359853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69352b5a-b974-4c0a-a5aa-a841ddde96e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:08.360131 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69352b5a-b974-4c0a-a5aa-a841ddde96e1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:08.367970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69352b5a-b974-4c0a-a5aa-a841ddde96e1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:08.369412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69352b5a-b974-4c0a-a5aa-a841ddde96e1 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:08.388094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-69352b5a-b974-4c0a-a5aa-a841ddde96e1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 22 11:33:08.388094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:08.388094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:08.388094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:08.388094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:08.388094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:08.388094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:08.388094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:08.388094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:08.388094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:08.388094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:08.388094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:08.388094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:33:08.388094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:33:08.388094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:33:08.388094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:33:08.390370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:08.390370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:08.390370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:33:08.390370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:08.390370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:33:08.390370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:08.390370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:08.390370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:08.390370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:08.390370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:08.390370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:08.390370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:08.390370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:08.390370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:08.390370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:08.390370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:08.390370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:08.390370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:08.391963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:08.391963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:08.391963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:33:08.391963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:33:08.391963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:33:08.391963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:33:08.391963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:33:08.391963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:33:08.391963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 22 11:33:08.391963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:08.391963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 444/890] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jun 22 11:33:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:08.391963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69352b5a-b974-4c0a-a5aa-a841ddde96e1 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:08.400808 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf6813-e0d3-45e3-be17-1118c4cc3d19 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:08.439246 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28bc408-d174-4cda-b454-dd6c1363e6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:08.440619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28bc408-d174-4cda-b454-dd6c1363e6c7 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:08.440905 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28bc408-d174-4cda-b454-dd6c1363e6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:08.441182 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28bc408-d174-4cda-b454-dd6c1363e6c7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:08.442310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28bc408-d174-4cda-b454-dd6c1363e6c7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:08.444441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28bc408-d174-4cda-b454-dd6c1363e6c7 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:08.461750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-e28bc408-d174-4cda-b454-dd6c1363e6c7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 22 11:33:08.461750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:08.461750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:33:08.461750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:33:08.461750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:33:08.461750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:33:08.461750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:33:08.461750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:08.461750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 22 11:33:08.461750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:08.461750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:08.461750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:08.461750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:08.461750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:08.461750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:08.461750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:08.461750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:08.462779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:08.462779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:08.462779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:08.462779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:08.462779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:08.462779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:08.462779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:33:08.462779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:33:08.462779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:33:08.462779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:33:08.462779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:08.462779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:08.462779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:08.462779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:08.462779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:08.462779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:08.464047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:08.464047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:08.464047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:08.464047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:08.464047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:08.464047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:08.464047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:08.464047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:08.464047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:08.464047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:08.464047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:33:08.464047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:08.464047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 22 11:33:08.464047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:08.464047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 447/891] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:33:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:08.465193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28bc408-d174-4cda-b454-dd6c1363e6c7 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:08.486335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:08.504990 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:08.512252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69352b5a-b974-4c0a-a5aa-a841ddde96e1 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:08.553982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a901a-6f9b-4b20-a6dc-12747a81aee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:08.554971 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a901a-6f9b-4b20-a6dc-12747a81aee9 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:08.555260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a901a-6f9b-4b20-a6dc-12747a81aee9 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:08.555839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a901a-6f9b-4b20-a6dc-12747a81aee9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:08.556645 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a901a-6f9b-4b20-a6dc-12747a81aee9 None admin] RBAC: Adding query filter params (interface=public, service_id=41a3e5fdc47d4e5880c5ae97593e762f, region_id=RegionOne) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:08.556959 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a901a-6f9b-4b20-a6dc-12747a81aee9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:08.558875 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a901a-6f9b-4b20-a6dc-12747a81aee9 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:08.569097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 445/892] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jun 22 11:33:08 2022] GET /identity/v3/endpoints?service_id=41a3e5fdc47d4e5880c5ae97593e762f&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:08.569922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a901a-6f9b-4b20-a6dc-12747a81aee9 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:08.578246 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28bc408-d174-4cda-b454-dd6c1363e6c7 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:08.612696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f962b-d70e-47ab-bd48-435c4bd0fdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:08.613290 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f962b-d70e-47ab-bd48-435c4bd0fdb7 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:08.613577 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f962b-d70e-47ab-bd48-435c4bd0fdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:08.613850 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f962b-d70e-47ab-bd48-435c4bd0fdb7 None admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:08.614683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f962b-d70e-47ab-bd48-435c4bd0fdb7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:08.614987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f962b-d70e-47ab-bd48-435c4bd0fdb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:08.616446 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f962b-d70e-47ab-bd48-435c4bd0fdb7 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:08.633858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 448/893] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jun 22 11:33:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:08.635114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed87c8-e4f7-4c17-8e46-8737d6b38bd4 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:08.635425 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed87c8-e4f7-4c17-8e46-8737d6b38bd4 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:08.635709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed87c8-e4f7-4c17-8e46-8737d6b38bd4 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:08.638615 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 449/894] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:08.639415 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed87c8-e4f7-4c17-8e46-8737d6b38bd4 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:08.670402 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a901a-6f9b-4b20-a6dc-12747a81aee9 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:08.703951 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83facaa2-8940-4496-8e12-1272ff4ee102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:08.704611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83facaa2-8940-4496-8e12-1272ff4ee102 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:08.704893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83facaa2-8940-4496-8e12-1272ff4ee102 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:08.705172 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83facaa2-8940-4496-8e12-1272ff4ee102 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:08.706458 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83facaa2-8940-4496-8e12-1272ff4ee102 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:08.706765 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83facaa2-8940-4496-8e12-1272ff4ee102 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:08.708158 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83facaa2-8940-4496-8e12-1272ff4ee102 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:08.714815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 446/895] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:33:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:08.715686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83facaa2-8940-4496-8e12-1272ff4ee102 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:08.719870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:08.741763 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:08.743972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed87c8-e4f7-4c17-8e46-8737d6b38bd4 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:08.779375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481ab7fc-b332-4385-9c4f-da7459d55248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:08.780066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ab7fc-b332-4385-9c4f-da7459d55248 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:08.780353 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ab7fc-b332-4385-9c4f-da7459d55248 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:08.780633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481ab7fc-b332-4385-9c4f-da7459d55248 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:08.781390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ab7fc-b332-4385-9c4f-da7459d55248 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:08.782690 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481ab7fc-b332-4385-9c4f-da7459d55248 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:08.788526 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 450/896] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jun 22 11:33:08 2022] GET /identity/v3/services => generated 1268 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:33:08.789845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e9967-93cb-4584-a1e8-8ef980f68eae None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:08.790135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e9967-93cb-4584-a1e8-8ef980f68eae None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:08.790409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e9967-93cb-4584-a1e8-8ef980f68eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:08.812752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-514e9967-93cb-4584-a1e8-8ef980f68eae None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:08.813986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-514e9967-93cb-4584-a1e8-8ef980f68eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:08.831318 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83facaa2-8940-4496-8e12-1272ff4ee102 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:08.870350 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85437d-7c88-49f9-a649-8bb416f8b10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:08.870931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85437d-7c88-49f9-a649-8bb416f8b10b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:08.871210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85437d-7c88-49f9-a649-8bb416f8b10b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:08.871516 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85437d-7c88-49f9-a649-8bb416f8b10b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:08.903909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 451/897] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:08 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:08.903909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e9967-93cb-4584-a1e8-8ef980f68eae None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:08.967243 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85437d-7c88-49f9-a649-8bb416f8b10b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:08.969800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85437d-7c88-49f9-a649-8bb416f8b10b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:08.994856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:09.002371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:09.040736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e9967-93cb-4584-a1e8-8ef980f68eae None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:09.078114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad3c64-f6c7-4d88-a0b1-ced401d5b98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:09.078780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad3c64-f6c7-4d88-a0b1-ced401d5b98e None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:09.079058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad3c64-f6c7-4d88-a0b1-ced401d5b98e None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:09.079329 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad3c64-f6c7-4d88-a0b1-ced401d5b98e None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:09.080055 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad3c64-f6c7-4d88-a0b1-ced401d5b98e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:09.080946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad3c64-f6c7-4d88-a0b1-ced401d5b98e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:09.082323 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad3c64-f6c7-4d88-a0b1-ced401d5b98e None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:09.092491 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 452/898] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:33:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:09.092491 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad3c64-f6c7-4d88-a0b1-ced401d5b98e None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:09.092761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 447/899] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:33:08 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 22 11:33:09.093576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85437d-7c88-49f9-a649-8bb416f8b10b None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:09.202061 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad3c64-f6c7-4d88-a0b1-ced401d5b98e None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:09.208265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85437d-7c88-49f9-a649-8bb416f8b10b None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:09.235129 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:09.238937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:09.241048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5102ebf-1e17-4202-add4-49250e3c80c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:09.242017 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5102ebf-1e17-4202-add4-49250e3c80c2 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:09.242236 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5102ebf-1e17-4202-add4-49250e3c80c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:09.242468 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5102ebf-1e17-4202-add4-49250e3c80c2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:09.243149 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5102ebf-1e17-4202-add4-49250e3c80c2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:09.244547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5102ebf-1e17-4202-add4-49250e3c80c2 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:09.255919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8aca0f-da54-4272-9394-a1f834871b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:09.256806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8aca0f-da54-4272-9394-a1f834871b0a None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:09.257255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8aca0f-da54-4272-9394-a1f834871b0a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:09.257671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-e5102ebf-1e17-4202-add4-49250e3c80c2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 22 11:33:09.257671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:09.257671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:33:09.257671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:33:09.257671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:33:09.257671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:33:09.257671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:33:09.257671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:09.257671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 22 11:33:09.257671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:09.257671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:09.257671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:09.257671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:09.257671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:09.257671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:09.257671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:09.257671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:09.258640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:09.258640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:09.258640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:09.258640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:09.258640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:09.258640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:09.258640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:33:09.258640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:33:09.258640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:33:09.258640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:33:09.258640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:09.258640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:09.258640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:09.258640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:09.258640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:09.258640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:09.259511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:09.259511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:09.259511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:09.259511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:09.259511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:09.259511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:09.259511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:09.259511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:09.259511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:09.259511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:09.259511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:33:09.259511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:09.259511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 22 11:33:09.259511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:09.259511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 453/900] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jun 22 11:33:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:09.260345 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5102ebf-1e17-4202-add4-49250e3c80c2 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:09.260570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8aca0f-da54-4272-9394-a1f834871b0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:09.261556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8aca0f-da54-4272-9394-a1f834871b0a None admin] RBAC: Adding query filter params (interface=public, service_id=15e98adb73824279a85b87cc76460c41, region_id=RegionOne) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:09.262032 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8aca0f-da54-4272-9394-a1f834871b0a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:09.265050 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8aca0f-da54-4272-9394-a1f834871b0a None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:09.273700 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 448/901] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jun 22 11:33:09 2022] GET /identity/v3/endpoints?service_id=15e98adb73824279a85b87cc76460c41&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:09.274786 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8aca0f-da54-4272-9394-a1f834871b0a None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:09.399116 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5102ebf-1e17-4202-add4-49250e3c80c2 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:09.402387 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8aca0f-da54-4272-9394-a1f834871b0a None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:09.434516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d3365-7b72-4843-9634-1d525f5b9069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:09.435410 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d3365-7b72-4843-9634-1d525f5b9069 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:09.435671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d3365-7b72-4843-9634-1d525f5b9069 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:09.435947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d3365-7b72-4843-9634-1d525f5b9069 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:09.449740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d3365-7b72-4843-9634-1d525f5b9069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:09.451254 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d3365-7b72-4843-9634-1d525f5b9069 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:09.454795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f977d2e-faf1-4ecf-8ab6-126361304a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:09.456202 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f977d2e-faf1-4ecf-8ab6-126361304a5a None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:09.456655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f977d2e-faf1-4ecf-8ab6-126361304a5a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:09.457017 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f977d2e-faf1-4ecf-8ab6-126361304a5a None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:09.458018 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f977d2e-faf1-4ecf-8ab6-126361304a5a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:09.458477 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f977d2e-faf1-4ecf-8ab6-126361304a5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:09.459889 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f977d2e-faf1-4ecf-8ab6-126361304a5a None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:09.465822 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 449/902] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jun 22 11:33:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:09.466647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f977d2e-faf1-4ecf-8ab6-126361304a5a None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:09.473692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-fd4d3365-7b72-4843-9634-1d525f5b9069 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:09.473692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:09.473692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:09.473692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:09.473692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:09.473692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:09.473692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:09.473692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:09.473692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:09.473692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:09.473692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:09.473692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:09.473692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:33:09.473692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:33:09.473692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:33:09.474746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:33:09.474746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:09.474746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:09.474746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:09.474746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:09.474746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:09.474746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:09.474746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:09.474746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:09.474746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:09.474746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:09.474746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:09.474746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:09.474746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:09.474746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:09.474746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:09.474746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:09.475778 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:33:09.475778 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:33:09.475778 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:09.475778 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:09.477217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 454/903] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:09.478110 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d3365-7b72-4843-9634-1d525f5b9069 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:09.534985 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:09.535892 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:09.588605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f977d2e-faf1-4ecf-8ab6-126361304a5a None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:09.599204 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d3365-7b72-4843-9634-1d525f5b9069 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:09.626023 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef282c-1a94-43cf-9a19-70441e08b188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:09.626740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef282c-1a94-43cf-9a19-70441e08b188 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:09.626978 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef282c-1a94-43cf-9a19-70441e08b188 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:09.627315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef282c-1a94-43cf-9a19-70441e08b188 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:09.628030 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef282c-1a94-43cf-9a19-70441e08b188 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:09.629396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef282c-1a94-43cf-9a19-70441e08b188 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:09.635080 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 450/904] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jun 22 11:33:09 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:33:09.635735 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d3cad-a323-4734-a802-75a2d971c746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:09.635897 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef282c-1a94-43cf-9a19-70441e08b188 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:09.636750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d3cad-a323-4734-a802-75a2d971c746 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:09.637087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d3cad-a323-4734-a802-75a2d971c746 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:09.637534 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d3cad-a323-4734-a802-75a2d971c746 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:09.638197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d3cad-a323-4734-a802-75a2d971c746 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:09.638577 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d3cad-a323-4734-a802-75a2d971c746 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:09.640346 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d3cad-a323-4734-a802-75a2d971c746 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:09.648935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 455/905] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jun 22 11:33:09 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:09.769870 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef282c-1a94-43cf-9a19-70441e08b188 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:09.788580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:09.816177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f4b52-28a2-4745-b097-42bc2fd16abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:09.817191 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f4b52-28a2-4745-b097-42bc2fd16abc None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:09.817631 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f4b52-28a2-4745-b097-42bc2fd16abc None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:09.818062 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f4b52-28a2-4745-b097-42bc2fd16abc None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:09.819707 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f4b52-28a2-4745-b097-42bc2fd16abc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:09.820261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f4b52-28a2-4745-b097-42bc2fd16abc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:09.821845 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f4b52-28a2-4745-b097-42bc2fd16abc None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:09.836104 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 451/906] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:09.841310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d3cad-a323-4734-a802-75a2d971c746 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:09.965333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d3cad-a323-4734-a802-75a2d971c746 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:09.979405 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:10.000558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aad63d-db1b-4015-9ef2-f6983574b265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:10.001355 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aad63d-db1b-4015-9ef2-f6983574b265 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:10.001636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aad63d-db1b-4015-9ef2-f6983574b265 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:10.001912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aad63d-db1b-4015-9ef2-f6983574b265 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:10.009445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aad63d-db1b-4015-9ef2-f6983574b265 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:10.010887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aad63d-db1b-4015-9ef2-f6983574b265 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:10.026045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-68aad63d-db1b-4015-9ef2-f6983574b265 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:10.026045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:10.026045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:10.026045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:10.026045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:10.026045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:10.026045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:10.026045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:10.026045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:10.026045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:10.026045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:10.026045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:10.026045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:33:10.026045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:33:10.026045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:33:10.026957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:33:10.026957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:10.026957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:10.026957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:10.026957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:10.026957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:10.026957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:10.026957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:10.026957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:10.026957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:10.026957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:10.026957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:10.026957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:10.026957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:10.026957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:10.026957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:10.026957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:10.027858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:33:10.027858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:33:10.027858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:33:10.027858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:33:10.027858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:33:10.027858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:33:10.027858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:10.027858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:10.027858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 456/907] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:33:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:10.032935 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f4b52-28a2-4745-b097-42bc2fd16abc None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:10.134549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f4b52-28a2-4745-b097-42bc2fd16abc None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:10.173471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9780d-d7ce-4677-a3f2-5d365c753f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:10.175092 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9780d-d7ce-4677-a3f2-5d365c753f27 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:10.175629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9780d-d7ce-4677-a3f2-5d365c753f27 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:10.176106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9780d-d7ce-4677-a3f2-5d365c753f27 None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:10.177119 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9780d-d7ce-4677-a3f2-5d365c753f27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:10.177625 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9780d-d7ce-4677-a3f2-5d365c753f27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:10.179250 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9780d-d7ce-4677-a3f2-5d365c753f27 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:10.191949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 452/908] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:33:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:10.197588 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aad63d-db1b-4015-9ef2-f6983574b265 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:10.301894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aad63d-db1b-4015-9ef2-f6983574b265 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:10.336033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c17bf5-fef4-4fbf-8c0a-3f6217a6b59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:10.336692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c17bf5-fef4-4fbf-8c0a-3f6217a6b59f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:10.336973 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c17bf5-fef4-4fbf-8c0a-3f6217a6b59f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:10.337246 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c17bf5-fef4-4fbf-8c0a-3f6217a6b59f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:10.338071 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c17bf5-fef4-4fbf-8c0a-3f6217a6b59f None admin] RBAC: Adding query filter params (role.id=8ce71a4c1e804871b92156359277b9ee, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=955c44ffe1bd426e900c7405df44d1cf) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:10.338378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c17bf5-fef4-4fbf-8c0a-3f6217a6b59f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:10.339827 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c17bf5-fef4-4fbf-8c0a-3f6217a6b59f None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:10.345726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 457/909] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:33:10 2022] GET /identity/v3/role_assignments?user.id=955c44ffe1bd426e900c7405df44d1cf&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=8ce71a4c1e804871b92156359277b9ee => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:10.902170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fca9f1-fa4b-4d69-b6d5-bf1cde37abc6 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:10.902693 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fca9f1-fa4b-4d69-b6d5-bf1cde37abc6 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:10.902768 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fca9f1-fa4b-4d69-b6d5-bf1cde37abc6 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:10.904363 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 453/910] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:10.909835 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867679dc-60d3-4d47-8f3a-162b6efba2de None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:10.910125 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867679dc-60d3-4d47-8f3a-162b6efba2de None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:10.910406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867679dc-60d3-4d47-8f3a-162b6efba2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:10.930846 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-867679dc-60d3-4d47-8f3a-162b6efba2de None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:10.932084 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-867679dc-60d3-4d47-8f3a-162b6efba2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:10.968122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:11.019550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 458/911] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:33:11.042067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fca9f1-fa4b-4d69-b6d5-bf1cde37abc6 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:11.078409 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:11.148638 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fca9f1-fa4b-4d69-b6d5-bf1cde37abc6 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:11.188842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e44f6-c919-429a-ac33-1d12845813ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:11.189650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e44f6-c919-429a-ac33-1d12845813ed None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:11.190525 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e44f6-c919-429a-ac33-1d12845813ed None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:11.190727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e44f6-c919-429a-ac33-1d12845813ed None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:11.191548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e44f6-c919-429a-ac33-1d12845813ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:11.192933 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e44f6-c919-429a-ac33-1d12845813ed None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:11.205821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-5d7e44f6-c919-429a-ac33-1d12845813ed None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 22 11:33:11.205821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:11.205821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:33:11.205821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:33:11.205821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:33:11.205821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:33:11.205821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:33:11.205821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:11.205821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 22 11:33:11.205821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:11.205821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:11.205821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:11.205821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:11.205821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:11.205821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:11.205821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:11.206955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:11.206955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:11.206955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:11.206955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:11.206955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:11.206955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:11.206955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:11.206955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:33:11.206955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:33:11.206955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:33:11.206955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:33:11.206955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:11.206955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:11.206955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:11.206955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:11.206955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:11.207964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:11.207964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:11.207964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:11.207964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:11.207964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:11.207964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:11.207964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:11.207964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:11.207964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:11.207964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:11.207964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:11.207964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:33:11.207964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:11.207964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 22 11:33:11.207964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:11.207964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 454/912] 10.222.0.36 () {56 vars in 1132 bytes} [Wed Jun 22 11:33:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:11.213343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867679dc-60d3-4d47-8f3a-162b6efba2de None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:11.326980 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867679dc-60d3-4d47-8f3a-162b6efba2de None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:11.366013 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d88c8-af59-44c6-b439-6e88c5f8b1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:11.366687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d88c8-af59-44c6-b439-6e88c5f8b1c1 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:11.366970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d88c8-af59-44c6-b439-6e88c5f8b1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:11.367247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d88c8-af59-44c6-b439-6e88c5f8b1c1 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:11.367984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d88c8-af59-44c6-b439-6e88c5f8b1c1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:11.368292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d88c8-af59-44c6-b439-6e88c5f8b1c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:11.369683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d88c8-af59-44c6-b439-6e88c5f8b1c1 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:11.377941 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 459/913] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jun 22 11:33:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:11.383197 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e44f6-c919-429a-ac33-1d12845813ed None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:11.500056 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e44f6-c919-429a-ac33-1d12845813ed None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:11.503560 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:11.538094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24eb6a9-82d8-44ed-9e7c-98028caa2345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:11.538964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24eb6a9-82d8-44ed-9e7c-98028caa2345 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:11.539246 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24eb6a9-82d8-44ed-9e7c-98028caa2345 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:11.539568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24eb6a9-82d8-44ed-9e7c-98028caa2345 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:11.540402 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24eb6a9-82d8-44ed-9e7c-98028caa2345 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:11.540753 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24eb6a9-82d8-44ed-9e7c-98028caa2345 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:11.542452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24eb6a9-82d8-44ed-9e7c-98028caa2345 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:11.549198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 455/914] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jun 22 11:33:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:11.554993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d88c8-af59-44c6-b439-6e88c5f8b1c1 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:11.595010 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:11.678144 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d88c8-af59-44c6-b439-6e88c5f8b1c1 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:11.718745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71dfee7-38df-438a-845e-9a628da61dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:11.719777 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71dfee7-38df-438a-845e-9a628da61dd5 None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:11.720209 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71dfee7-38df-438a-845e-9a628da61dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:11.720664 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71dfee7-38df-438a-845e-9a628da61dd5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:11.722186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71dfee7-38df-438a-845e-9a628da61dd5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:11.724519 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71dfee7-38df-438a-845e-9a628da61dd5 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:11.745942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 460/915] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jun 22 11:33:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:33:11.893975 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0138e9-02ce-477f-acfd-2ac8c3acae62 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:11.894261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0138e9-02ce-477f-acfd-2ac8c3acae62 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:11.894551 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0138e9-02ce-477f-acfd-2ac8c3acae62 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:11.896247 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 456/916] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:11.901724 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2295d7-b512-4e19-86b7-77653bc46197 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:11.902080 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2295d7-b512-4e19-86b7-77653bc46197 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:11.902288 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2295d7-b512-4e19-86b7-77653bc46197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:11.919985 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-5d2295d7-b512-4e19-86b7-77653bc46197 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:11.921391 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5d2295d7-b512-4e19-86b7-77653bc46197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:11.944736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a469184-4096-4188-b29a-8993c22f3f5b None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:11.945210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a469184-4096-4188-b29a-8993c22f3f5b None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:11.945311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a469184-4096-4188-b29a-8993c22f3f5b None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:11.947880 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 457/917] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:33:11.952924 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12f990-a12f-4c28-9641-dd126511c26e None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:11.953221 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12f990-a12f-4c28-9641-dd126511c26e None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:11.953494 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12f990-a12f-4c28-9641-dd126511c26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:11.972847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-cb12f990-a12f-4c28-9641-dd126511c26e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:11.974043 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-cb12f990-a12f-4c28-9641-dd126511c26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:12.012663 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 461/918] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:12.037270 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2295d7-b512-4e19-86b7-77653bc46197 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:12.055837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:12.056106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 458/919] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:12.078831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12f990-a12f-4c28-9641-dd126511c26e None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:12.148989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2295d7-b512-4e19-86b7-77653bc46197 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:12.188345 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ac45d-53f3-403d-aca3-c1b38fb4813d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:12.189746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ac45d-53f3-403d-aca3-c1b38fb4813d None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:12.191023 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ac45d-53f3-403d-aca3-c1b38fb4813d None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:12.191477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ac45d-53f3-403d-aca3-c1b38fb4813d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:12.192290 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ac45d-53f3-403d-aca3-c1b38fb4813d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:12.194391 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ac45d-53f3-403d-aca3-c1b38fb4813d None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:12.197446 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:12.207447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12f990-a12f-4c28-9641-dd126511c26e None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:12.213475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-0c4ac45d-53f3-403d-aca3-c1b38fb4813d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 22 11:33:12.213475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:12.213475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:33:12.213475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:33:12.213475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:33:12.213475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:33:12.213475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:33:12.213475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:12.213475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 22 11:33:12.213475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:12.213475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:12.213475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:12.213475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:12.213475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:12.213475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:12.213475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:12.213475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:12.214666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:12.214666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:12.214666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:12.214666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:12.214666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:12.214666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:12.214666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:33:12.214666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:33:12.214666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:33:12.214666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:33:12.214666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:12.214666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:12.214666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:12.214666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:12.214666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:12.214666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:12.215593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:12.215593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:12.215593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:12.215593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:12.215593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:12.215593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:12.215593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:12.215593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:12.215593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:12.215593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:12.215593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:33:12.215593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:12.215593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 22 11:33:12.215593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:12.218660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 462/920] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:33:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:12.226032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ac45d-53f3-403d-aca3-c1b38fb4813d None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:12.258565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078aeee9-234e-423a-ab7c-5efbd2257f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:12.259528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078aeee9-234e-423a-ab7c-5efbd2257f11 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:12.259962 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078aeee9-234e-423a-ab7c-5efbd2257f11 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:12.260388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078aeee9-234e-423a-ab7c-5efbd2257f11 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:12.261296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078aeee9-234e-423a-ab7c-5efbd2257f11 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:12.262836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078aeee9-234e-423a-ab7c-5efbd2257f11 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:12.278171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 459/921] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jun 22 11:33:12 2022] POST /identity/v3/services => generated 241 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:33:12.327908 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:12.355247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ac45d-53f3-403d-aca3-c1b38fb4813d None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:12.400978 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81cb9f-440e-4aa3-98bb-ad518efa8589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:12.401577 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81cb9f-440e-4aa3-98bb-ad518efa8589 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:12.401863 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81cb9f-440e-4aa3-98bb-ad518efa8589 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:12.402144 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81cb9f-440e-4aa3-98bb-ad518efa8589 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:12.402854 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81cb9f-440e-4aa3-98bb-ad518efa8589 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:12.403166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81cb9f-440e-4aa3-98bb-ad518efa8589 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:12.404590 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81cb9f-440e-4aa3-98bb-ad518efa8589 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:12.411250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 463/922] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:33:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:12.417210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078aeee9-234e-423a-ab7c-5efbd2257f11 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:12.535057 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078aeee9-234e-423a-ab7c-5efbd2257f11 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:12.567113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a7a61-05d1-4839-bd6c-f12dc93f5c60 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:12.567460 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a7a61-05d1-4839-bd6c-f12dc93f5c60 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:12.569508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a7a61-05d1-4839-bd6c-f12dc93f5c60 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:12.569508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 464/923] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:12.576205 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55349e2f-0c7f-4425-b9da-6849459bb550 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:12.576205 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55349e2f-0c7f-4425-b9da-6849459bb550 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:12.576205 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55349e2f-0c7f-4425-b9da-6849459bb550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:12.576688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb6b9e-5081-448f-a58b-0e456c278428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:12.577325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb6b9e-5081-448f-a58b-0e456c278428 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:12.577612 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb6b9e-5081-448f-a58b-0e456c278428 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:12.577928 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb6b9e-5081-448f-a58b-0e456c278428 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:12.578650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb6b9e-5081-448f-a58b-0e456c278428 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:12.578961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb6b9e-5081-448f-a58b-0e456c278428 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:12.580685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb6b9e-5081-448f-a58b-0e456c278428 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:12.586711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 460/924] 10.222.0.36 () {56 vars in 1114 bytes} [Wed Jun 22 11:33:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:12.592497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb6b9e-5081-448f-a58b-0e456c278428 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:12.597759 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-55349e2f-0c7f-4425-b9da-6849459bb550 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:12.599514 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-55349e2f-0c7f-4425-b9da-6849459bb550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:12.614595 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:12.692461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 465/925] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:33:12.716063 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55349e2f-0c7f-4425-b9da-6849459bb550 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:12.727261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb6b9e-5081-448f-a58b-0e456c278428 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:12.776252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791b99c-8fc1-4b21-958b-350a480dc104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:12.776920 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791b99c-8fc1-4b21-958b-350a480dc104 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:12.777206 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791b99c-8fc1-4b21-958b-350a480dc104 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:12.777487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791b99c-8fc1-4b21-958b-350a480dc104 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:12.779671 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791b99c-8fc1-4b21-958b-350a480dc104 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:12.781050 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791b99c-8fc1-4b21-958b-350a480dc104 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:12.809173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:12.811934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 461/926] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jun 22 11:33:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:33:12.856127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55349e2f-0c7f-4425-b9da-6849459bb550 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:12.903698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647cdec-4830-48f0-afa1-7011891ad930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:12.904643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647cdec-4830-48f0-afa1-7011891ad930 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:12.904929 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647cdec-4830-48f0-afa1-7011891ad930 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:12.905208 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647cdec-4830-48f0-afa1-7011891ad930 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:12.918175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647cdec-4830-48f0-afa1-7011891ad930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:33:12.932599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-5647cdec-4830-48f0-afa1-7011891ad930 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 22 11:33:12.932599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:12.932599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:12.932599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:12.932599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:12.932599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:12.932599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:12.932599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:12.932599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:12.932599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:12.932599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:12.932599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:12.932599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:33:12.932599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:33:12.932599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:33:12.932599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:33:12.933599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:33:12.933599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:33:12.933599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:33:12.933599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:33:12.933599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:12.933599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:12.933599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:12.933599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:12.933599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:12.933599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:12.933599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:12.933599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:12.933599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:12.933599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:12.933599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:12.933599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:12.934437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:12.934437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:12.934437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:12.934437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:12.934437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:33:12.934437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:33:12.934437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:33:12.934437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:33:12.934437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:33:12.934437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:33:12.934437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 22 11:33:12.934437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:12.934437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 466/927] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jun 22 11:33:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:12.943639 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791b99c-8fc1-4b21-958b-350a480dc104 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:13.026826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:13.061538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791b99c-8fc1-4b21-958b-350a480dc104 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:13.097066 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83a81e-1f6b-4a86-b92a-68d45ce59978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:13.097740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83a81e-1f6b-4a86-b92a-68d45ce59978 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:13.098022 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83a81e-1f6b-4a86-b92a-68d45ce59978 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:13.098297 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83a81e-1f6b-4a86-b92a-68d45ce59978 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:13.099053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83a81e-1f6b-4a86-b92a-68d45ce59978 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:13.099379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83a81e-1f6b-4a86-b92a-68d45ce59978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:13.100750 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83a81e-1f6b-4a86-b92a-68d45ce59978 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:13.109990 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 462/928] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jun 22 11:33:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:13.116162 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647cdec-4830-48f0-afa1-7011891ad930 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:13.227000 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647cdec-4830-48f0-afa1-7011891ad930 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:13.250559 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:13.262834 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4723a9-4dbe-40b1-a32e-962083f70903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:13.263670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4723a9-4dbe-40b1-a32e-962083f70903 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:13.263958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4723a9-4dbe-40b1-a32e-962083f70903 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:13.264234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4723a9-4dbe-40b1-a32e-962083f70903 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:13.275755 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4723a9-4dbe-40b1-a32e-962083f70903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:13.277220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4723a9-4dbe-40b1-a32e-962083f70903 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:13.294706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-2d4723a9-4dbe-40b1-a32e-962083f70903 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:13.294706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:13.294706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:13.294706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:13.294706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:13.294706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:13.294706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:13.294706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:13.294706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:13.294706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:13.294706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:13.294706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:13.294706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:33:13.294706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:33:13.294706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:33:13.295644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:33:13.295644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:13.295644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:13.295644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:13.295644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:13.295644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:13.295644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:13.295644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:13.295644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:13.295644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:13.295644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:13.295644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:13.295644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:13.295644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:13.295644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:13.295644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:13.295644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:13.296687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:33:13.296687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:33:13.296687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:13.296687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:13.296687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 467/929] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:13.304444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83a81e-1f6b-4a86-b92a-68d45ce59978 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:13.413363 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83a81e-1f6b-4a86-b92a-68d45ce59978 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:13.450699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91176948-04fc-4321-9e40-20efa28c163b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:13.451842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91176948-04fc-4321-9e40-20efa28c163b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:13.452380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91176948-04fc-4321-9e40-20efa28c163b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:13.452869 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91176948-04fc-4321-9e40-20efa28c163b None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:13.453794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91176948-04fc-4321-9e40-20efa28c163b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:13.454310 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91176948-04fc-4321-9e40-20efa28c163b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:13.455951 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91176948-04fc-4321-9e40-20efa28c163b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:13.457806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:13.471720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 463/930] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:13.477596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4723a9-4dbe-40b1-a32e-962083f70903 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:13.587010 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4723a9-4dbe-40b1-a32e-962083f70903 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:13.628111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d957b1-4c55-4039-9011-979c25a6b93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:13.628936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d957b1-4c55-4039-9011-979c25a6b93a None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:13.629224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d957b1-4c55-4039-9011-979c25a6b93a None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:13.629505 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d957b1-4c55-4039-9011-979c25a6b93a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:13.639088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d957b1-4c55-4039-9011-979c25a6b93a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:13.641303 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d957b1-4c55-4039-9011-979c25a6b93a None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:13.657114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-53d957b1-4c55-4039-9011-979c25a6b93a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 22 11:33:13.657114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:13.657114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:13.657114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:13.657114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:13.657114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:13.657114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:13.657114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:13.657114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:13.657114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:13.657114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:13.657114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:13.657114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:33:13.657114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:33:13.657114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:33:13.657114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:33:13.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:13.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:13.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:33:13.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:13.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:33:13.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:13.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:13.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:13.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:13.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:13.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:13.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:13.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:13.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:13.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:13.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:13.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:13.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:13.659124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:13.659124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:13.659124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:33:13.659124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:33:13.659124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:33:13.659124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:33:13.659124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:33:13.659124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:33:13.659124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 22 11:33:13.659124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:13.663071 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 468/931] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jun 22 11:33:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:13.668459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91176948-04fc-4321-9e40-20efa28c163b None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:13.772031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91176948-04fc-4321-9e40-20efa28c163b None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:13.807059 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3942c-d37d-4a8f-9da1-27a413eb6f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:13.807756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3942c-d37d-4a8f-9da1-27a413eb6f09 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:13.808035 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3942c-d37d-4a8f-9da1-27a413eb6f09 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:13.808308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3942c-d37d-4a8f-9da1-27a413eb6f09 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:13.809134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3942c-d37d-4a8f-9da1-27a413eb6f09 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:13.809438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3942c-d37d-4a8f-9da1-27a413eb6f09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:13.810845 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe3942c-d37d-4a8f-9da1-27a413eb6f09 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:13.828668 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 464/932] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jun 22 11:33:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:13.834157 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d957b1-4c55-4039-9011-979c25a6b93a None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:13.870298 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:13.940752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d957b1-4c55-4039-9011-979c25a6b93a None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:13.978670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e07fc-0aaa-4814-9c35-3cd387823b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:13.979506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e07fc-0aaa-4814-9c35-3cd387823b57 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:13.979789 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e07fc-0aaa-4814-9c35-3cd387823b57 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:13.980066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e07fc-0aaa-4814-9c35-3cd387823b57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:13.992098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e07fc-0aaa-4814-9c35-3cd387823b57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:13.992292 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2b614-2555-4c51-a43f-4f3bab7251e2 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:13.992530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2b614-2555-4c51-a43f-4f3bab7251e2 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:13.992764 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2b614-2555-4c51-a43f-4f3bab7251e2 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:13.993569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e07fc-0aaa-4814-9c35-3cd387823b57 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:13.994358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 465/933] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:33:14.000728 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d156b7-7d34-4cfb-8585-7f0ecb88fb99 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:14.001132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d156b7-7d34-4cfb-8585-7f0ecb88fb99 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:14.001531 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d156b7-7d34-4cfb-8585-7f0ecb88fb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:14.015947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-5d5e07fc-0aaa-4814-9c35-3cd387823b57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:14.015947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:14.015947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:14.015947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:14.015947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:14.015947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:14.015947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:14.015947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:14.015947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:14.015947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:14.015947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:14.015947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:14.015947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:33:14.015947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:33:14.015947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:33:14.016999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:33:14.016999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:14.016999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:14.016999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:14.016999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:14.016999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:14.016999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:14.016999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:14.016999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:14.016999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:14.016999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:14.016999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:14.016999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:14.016999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:14.016999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:14.016999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:14.016999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:14.017973 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:33:14.017973 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:33:14.017973 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:14.017973 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:14.017973 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 469/934] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:14.025703 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-f8d156b7-7d34-4cfb-8585-7f0ecb88fb99 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:14.026835 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f8d156b7-7d34-4cfb-8585-7f0ecb88fb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:14.027099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e07fc-0aaa-4814-9c35-3cd387823b57 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:14.092026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:14.122633 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 466/935] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:14.136192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:14.145325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d156b7-7d34-4cfb-8585-7f0ecb88fb99 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:14.156287 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e07fc-0aaa-4814-9c35-3cd387823b57 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:14.198493 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24d94b-aa36-4139-bb52-56fdc93c2b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:14.199672 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24d94b-aa36-4139-bb52-56fdc93c2b64 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:14.200174 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24d94b-aa36-4139-bb52-56fdc93c2b64 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:14.200688 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24d94b-aa36-4139-bb52-56fdc93c2b64 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:14.201623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24d94b-aa36-4139-bb52-56fdc93c2b64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:14.202228 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24d94b-aa36-4139-bb52-56fdc93c2b64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:14.203975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24d94b-aa36-4139-bb52-56fdc93c2b64 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:14.218420 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 470/936] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:14.224668 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24d94b-aa36-4139-bb52-56fdc93c2b64 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:14.254199 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d156b7-7d34-4cfb-8585-7f0ecb88fb99 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:14.290047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55d623-58c5-47b9-bd5c-79169765ee3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:14.290756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55d623-58c5-47b9-bd5c-79169765ee3d None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:14.291038 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55d623-58c5-47b9-bd5c-79169765ee3d None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:14.291311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55d623-58c5-47b9-bd5c-79169765ee3d None admin] PATH_INFO: `/v3/services/compute` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:14.292054 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55d623-58c5-47b9-bd5c-79169765ee3d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:14.293352 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55d623-58c5-47b9-bd5c-79169765ee3d None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:14.305580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-ec55d623-58c5-47b9-bd5c-79169765ee3d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 22 11:33:14.305580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:14.305580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:33:14.305580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:33:14.305580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:33:14.305580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:33:14.305580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:33:14.305580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:14.305580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 22 11:33:14.305580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:14.305580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:14.305580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:14.305580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:14.305580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:14.305580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:14.305580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:14.305580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:14.306589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:14.306589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:14.306589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:14.306589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:14.306589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:14.306589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:14.306589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:33:14.306589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:33:14.306589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:33:14.306589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:33:14.306589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:14.306589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:14.306589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:14.306589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:14.306589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:14.306589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:14.307461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:14.307461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:14.307461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:14.307461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:14.307461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:14.307461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:14.307461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:14.307461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:14.307461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:14.307461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:14.307461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:33:14.307461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:14.307461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 22 11:33:14.307461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:14.311265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 467/937] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:33:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:14.312163 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55d623-58c5-47b9-bd5c-79169765ee3d None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:14.343835 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24d94b-aa36-4139-bb52-56fdc93c2b64 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:14.359598 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:14.386534 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be6426-3b7e-4637-92af-06bf2cce5a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:14.387790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be6426-3b7e-4637-92af-06bf2cce5a76 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:14.388294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be6426-3b7e-4637-92af-06bf2cce5a76 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:14.388912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be6426-3b7e-4637-92af-06bf2cce5a76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:14.398228 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be6426-3b7e-4637-92af-06bf2cce5a76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:14.400200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be6426-3b7e-4637-92af-06bf2cce5a76 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:14.415865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55d623-58c5-47b9-bd5c-79169765ee3d None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:14.416422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-89be6426-3b7e-4637-92af-06bf2cce5a76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:14.416422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:14.416422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:14.416422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:14.416422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:14.416422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:14.416422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:14.416422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:14.416422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:14.416422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:14.416422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:14.416422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:14.416422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:33:14.416422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:33:14.416422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:33:14.417299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:33:14.417299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:14.417299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:14.417299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:14.417299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:14.417299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:14.417299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:14.417299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:14.417299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:14.417299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:14.417299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:14.417299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:14.417299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:14.417299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:14.417299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:14.417299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:14.417299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:14.418217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:33:14.418217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:33:14.418217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:33:14.418217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:33:14.418217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:33:14.418217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:33:14.418217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:14.418217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:14.421759 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 471/938] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:33:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:14.429523 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be6426-3b7e-4637-92af-06bf2cce5a76 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:14.443633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:14.461750 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119b271-f927-4f84-b120-807f50304c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:14.463034 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119b271-f927-4f84-b120-807f50304c45 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:14.463822 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119b271-f927-4f84-b120-807f50304c45 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:14.464731 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119b271-f927-4f84-b120-807f50304c45 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:14.465967 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119b271-f927-4f84-b120-807f50304c45 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:14.466505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119b271-f927-4f84-b120-807f50304c45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:14.469017 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a119b271-f927-4f84-b120-807f50304c45 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:14.477370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 468/939] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jun 22 11:33:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:14.483090 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119b271-f927-4f84-b120-807f50304c45 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:14.555837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be6426-3b7e-4637-92af-06bf2cce5a76 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:14.595581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cdf33-4ea7-4054-a925-4ceb92bcbe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:14.596784 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cdf33-4ea7-4054-a925-4ceb92bcbe2b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:14.597061 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cdf33-4ea7-4054-a925-4ceb92bcbe2b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:14.597335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cdf33-4ea7-4054-a925-4ceb92bcbe2b None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:14.598119 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cdf33-4ea7-4054-a925-4ceb92bcbe2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:14.598426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cdf33-4ea7-4054-a925-4ceb92bcbe2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:14.599888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cdf33-4ea7-4054-a925-4ceb92bcbe2b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:14.608876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119b271-f927-4f84-b120-807f50304c45 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:14.618566 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 472/940] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:33:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:14.618566 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c66995-a667-49df-9a97-631c8dedcb3b None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:14.618566 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c66995-a667-49df-9a97-631c8dedcb3b None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:14.618566 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c66995-a667-49df-9a97-631c8dedcb3b None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:14.618566 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 473/941] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:14.620192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c66995-a667-49df-9a97-631c8dedcb3b None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:14.624014 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:14.649458 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794cb2e-69fe-432e-a1d1-8f0157682a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:14.650141 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794cb2e-69fe-432e-a1d1-8f0157682a7f None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:14.650431 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794cb2e-69fe-432e-a1d1-8f0157682a7f None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:14.650709 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794cb2e-69fe-432e-a1d1-8f0157682a7f None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:14.651447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794cb2e-69fe-432e-a1d1-8f0157682a7f None admin] RBAC: Adding query filter params (type=compute) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:14.651756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794cb2e-69fe-432e-a1d1-8f0157682a7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:14.653114 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794cb2e-69fe-432e-a1d1-8f0157682a7f None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:14.658109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 469/942] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jun 22 11:33:14 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:14.660211 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb8fca-79d4-4734-91c5-7961027747a4 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:14.660499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb8fca-79d4-4734-91c5-7961027747a4 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:14.660774 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb8fca-79d4-4734-91c5-7961027747a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:14.680125 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-3ccb8fca-79d4-4734-91c5-7961027747a4 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:14.681359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-3ccb8fca-79d4-4734-91c5-7961027747a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:14.732331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:14.763227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 470/943] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:14.787593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb8fca-79d4-4734-91c5-7961027747a4 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:14.798120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c66995-a667-49df-9a97-631c8dedcb3b None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:14.839185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652067c-330d-454f-a661-d9a3c1144ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:14.840672 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652067c-330d-454f-a661-d9a3c1144ceb None admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:14.841166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652067c-330d-454f-a661-d9a3c1144ceb None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:14.841679 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652067c-330d-454f-a661-d9a3c1144ceb None admin] PATH_INFO: `/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/955c44ffe1bd426e900c7405df44d1cf/roles/8ce71a4c1e804871b92156359277b9ee` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:14.844616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652067c-330d-454f-a661-d9a3c1144ceb None admin] RBAC: Authorizing `identity:create_grant(project_id=47d2bb094c9843c9a7726f1d672d2433, user_id=955c44ffe1bd426e900c7405df44d1cf, role_id=8ce71a4c1e804871b92156359277b9ee)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:14.846831 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652067c-330d-454f-a661-d9a3c1144ceb None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:14.871571 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 474/944] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:33:14 2022] PUT /identity/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/955c44ffe1bd426e900c7405df44d1cf/roles/8ce71a4c1e804871b92156359277b9ee => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:33:14.910835 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb8fca-79d4-4734-91c5-7961027747a4 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:14.913812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:14.960133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da39820-7fa3-4d14-a574-24e21f2687f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:14.961075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da39820-7fa3-4d14-a574-24e21f2687f3 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:14.961345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da39820-7fa3-4d14-a574-24e21f2687f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:14.961626 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da39820-7fa3-4d14-a574-24e21f2687f3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:14.962347 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da39820-7fa3-4d14-a574-24e21f2687f3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:14.963844 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da39820-7fa3-4d14-a574-24e21f2687f3 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:14.977877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-7da39820-7fa3-4d14-a574-24e21f2687f3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 22 11:33:14.977877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:14.977877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:33:14.977877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:33:14.977877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:33:14.977877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:33:14.977877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:33:14.977877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:14.977877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 22 11:33:14.977877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:14.977877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:14.977877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:14.977877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:14.977877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:14.977877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:14.977877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:14.977877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:14.978981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:14.978981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:14.978981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:14.978981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:14.978981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:14.978981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:14.978981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:33:14.978981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:33:14.978981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:33:14.978981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:33:14.978981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:14.978981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:14.978981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:14.978981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:14.978981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:14.978981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:14.979847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:14.979847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:14.979847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:14.979847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:14.979847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:14.979847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:14.979847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:14.979847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:14.979847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:14.979847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:14.979847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:33:14.979847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:14.979847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 22 11:33:14.979847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:14.983456 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 471/945] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jun 22 11:33:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:14.988512 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652067c-330d-454f-a661-d9a3c1144ceb None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:15.027255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b3820-f9ac-4ee8-b4f0-e6a49165d699 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:15.027598 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b3820-f9ac-4ee8-b4f0-e6a49165d699 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:15.027833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b3820-f9ac-4ee8-b4f0-e6a49165d699 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:15.029664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 472/946] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:15.036159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690ef63-2c27-4cfa-98a5-ed21eae828f7 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:15.036445 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690ef63-2c27-4cfa-98a5-ed21eae828f7 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:15.036712 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690ef63-2c27-4cfa-98a5-ed21eae828f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:15.058085 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-6690ef63-2c27-4cfa-98a5-ed21eae828f7 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:15.059294 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-6690ef63-2c27-4cfa-98a5-ed21eae828f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:15.122315 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652067c-330d-454f-a661-d9a3c1144ceb None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:15.138853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 473/947] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:33:15.157181 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:15.168201 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad458c-94a6-42f1-b9df-a30e0ab6f15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:15.169288 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad458c-94a6-42f1-b9df-a30e0ab6f15e None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:15.169799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad458c-94a6-42f1-b9df-a30e0ab6f15e None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:15.171368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6690ef63-2c27-4cfa-98a5-ed21eae828f7 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:15.171710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad458c-94a6-42f1-b9df-a30e0ab6f15e None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:15.172495 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad458c-94a6-42f1-b9df-a30e0ab6f15e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:15.172821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad458c-94a6-42f1-b9df-a30e0ab6f15e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:15.174299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad458c-94a6-42f1-b9df-a30e0ab6f15e None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:15.181453 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 475/948] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jun 22 11:33:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:15.186777 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad458c-94a6-42f1-b9df-a30e0ab6f15e None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:15.278586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:15.283049 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6690ef63-2c27-4cfa-98a5-ed21eae828f7 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:15.297706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad458c-94a6-42f1-b9df-a30e0ab6f15e None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:15.320647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92036ef0-dc4f-44d9-b43d-9b63f543beb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:15.321683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92036ef0-dc4f-44d9-b43d-9b63f543beb7 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:15.322178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92036ef0-dc4f-44d9-b43d-9b63f543beb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:15.322687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92036ef0-dc4f-44d9-b43d-9b63f543beb7 None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:15.323664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92036ef0-dc4f-44d9-b43d-9b63f543beb7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:15.325320 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92036ef0-dc4f-44d9-b43d-9b63f543beb7 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:15.326923 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: WARNING keystone.server.flask.common [None req-92036ef0-dc4f-44d9-b43d-9b63f543beb7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 22 11:33:15.334053 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407a77b-4c00-4bb1-bb0e-3cce80106925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:15.334720 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407a77b-4c00-4bb1-bb0e-3cce80106925 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:15.335004 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407a77b-4c00-4bb1-bb0e-3cce80106925 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:15.335281 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407a77b-4c00-4bb1-bb0e-3cce80106925 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:15.336035 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407a77b-4c00-4bb1-bb0e-3cce80106925 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:15.336345 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407a77b-4c00-4bb1-bb0e-3cce80106925 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:15.337807 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407a77b-4c00-4bb1-bb0e-3cce80106925 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:15.347427 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 476/949] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jun 22 11:33:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:15.349902 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407a77b-4c00-4bb1-bb0e-3cce80106925 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:15.350476 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.sql.core [None req-92036ef0-dc4f-44d9-b43d-9b63f543beb7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 22 11:33:15.350476 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 22 11:33:15.350476 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [parameters: {'id': 'c28087e1689c424b87717f8db31638a7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 22 11:33:15.350476 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86768) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 22 11:33:15.369878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-92036ef0-dc4f-44d9-b43d-9b63f543beb7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c28087e1689c424b87717f8db31638a7.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c28087e1689c424b87717f8db31638a7. Jun 22 11:33:15.369878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:15.369878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 22 11:33:15.369878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 22 11:33:15.369878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 22 11:33:15.369878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 22 11:33:15.369878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 22 11:33:15.369878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application result = self._query(query) Jun 22 11:33:15.369878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 22 11:33:15.369878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application conn.query(q) Jun 22 11:33:15.369878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 22 11:33:15.369878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 22 11:33:15.369878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 22 11:33:15.371471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application result.read() Jun 22 11:33:15.371471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 22 11:33:15.371471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 22 11:33:15.371471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 22 11:33:15.371471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 22 11:33:15.371471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 22 11:33:15.371471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 22 11:33:15.371471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 22 11:33:15.371471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 22 11:33:15.371471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 22 11:33:15.371471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:15.371471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 22 11:33:15.371471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:15.371471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:15.371471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 22 11:33:15.371471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 22 11:33:15.371471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 22 11:33:15.371471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 22 11:33:15.373302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 22 11:33:15.373302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application next(self.gen) Jun 22 11:33:15.373302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 22 11:33:15.373302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application yield resource Jun 22 11:33:15.373302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 22 11:33:15.373302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application next(self.gen) Jun 22 11:33:15.373302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 22 11:33:15.373302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self.session.rollback() Jun 22 11:33:15.373302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 22 11:33:15.373302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self.force_reraise() Jun 22 11:33:15.373302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 22 11:33:15.373302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise self.value Jun 22 11:33:15.373302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 22 11:33:15.373302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 22 11:33:15.373302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 22 11:33:15.373302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application session.commit() Jun 22 11:33:15.373302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 22 11:33:15.375031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 22 11:33:15.375031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 22 11:33:15.375031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._prepare_impl() Jun 22 11:33:15.375031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 22 11:33:15.375031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self.session.flush() Jun 22 11:33:15.375031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 22 11:33:15.375031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._flush(objects) Jun 22 11:33:15.375031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 22 11:33:15.375031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 22 11:33:15.375031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 22 11:33:15.375031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application compat.raise_( Jun 22 11:33:15.375031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 22 11:33:15.375031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception Jun 22 11:33:15.375031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 22 11:33:15.375031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application flush_context.execute() Jun 22 11:33:15.375031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 22 11:33:15.375031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rec.execute(self) Jun 22 11:33:15.379177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 22 11:33:15.379177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 22 11:33:15.379177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 22 11:33:15.379177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 22 11:33:15.379177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 22 11:33:15.379177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 22 11:33:15.379177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 22 11:33:15.379177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 22 11:33:15.379177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 22 11:33:15.379177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 22 11:33:15.379177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 22 11:33:15.379177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 22 11:33:15.379177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 22 11:33:15.379177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 22 11:33:15.379177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 22 11:33:15.380623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 22 11:33:15.380623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 22 11:33:15.380623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception Jun 22 11:33:15.380623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 22 11:33:15.380623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 22 11:33:15.380623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 22 11:33:15.380623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 22 11:33:15.380623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 22 11:33:15.380623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application result = self._query(query) Jun 22 11:33:15.380623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 22 11:33:15.380623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application conn.query(q) Jun 22 11:33:15.380623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 22 11:33:15.380623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 22 11:33:15.380623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 22 11:33:15.380623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application result.read() Jun 22 11:33:15.380623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 22 11:33:15.380623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 22 11:33:15.382260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 22 11:33:15.382260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 22 11:33:15.382260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 22 11:33:15.382260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 22 11:33:15.382260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 22 11:33:15.382260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 22 11:33:15.382260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 22 11:33:15.382260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 22 11:33:15.382260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [parameters: {'id': 'c28087e1689c424b87717f8db31638a7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 22 11:33:15.382260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 22 11:33:15.382260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:15.382260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:15.382260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:15.382260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:15.382260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 22 11:33:15.383842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 22 11:33:15.383842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 22 11:33:15.383842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 22 11:33:15.383842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 22 11:33:15.383842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:15.383842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:15.383842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:15.383842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:15.383842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:15.383842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:15.383842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:15.383842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:15.383842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:15.383842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:15.383842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:15.383842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:15.383842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:15.385611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:15.385611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 22 11:33:15.385611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 22 11:33:15.385611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:15.385611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:15.385611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 22 11:33:15.385611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 22 11:33:15.385611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c28087e1689c424b87717f8db31638a7. Jun 22 11:33:15.385611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:15.397644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 474/950] 10.222.0.36 () {60 vars in 1140 bytes} [Wed Jun 22 11:33:15 2022] POST /identity/v3/projects => generated 262 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:15.399803 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92036ef0-dc4f-44d9-b43d-9b63f543beb7 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:15.419738 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:15.458888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407a77b-4c00-4bb1-bb0e-3cce80106925 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:15.499087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d79ded-9d9e-4a62-a369-15aaa353a0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:15.500128 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d79ded-9d9e-4a62-a369-15aaa353a0b4 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:15.500555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d79ded-9d9e-4a62-a369-15aaa353a0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:15.501010 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d79ded-9d9e-4a62-a369-15aaa353a0b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:15.501949 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d79ded-9d9e-4a62-a369-15aaa353a0b4 None admin] RBAC: Adding query filter params (interface=public, service_id=574162db2ef04550a1a686abcb6db4ef, region_id=RegionOne) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:15.502425 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d79ded-9d9e-4a62-a369-15aaa353a0b4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:15.504029 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d79ded-9d9e-4a62-a369-15aaa353a0b4 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:15.510875 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 477/951] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jun 22 11:33:15 2022] GET /identity/v3/endpoints?service_id=574162db2ef04550a1a686abcb6db4ef&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:15.515844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d79ded-9d9e-4a62-a369-15aaa353a0b4 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:15.563491 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92036ef0-dc4f-44d9-b43d-9b63f543beb7 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:15.609931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c61847-dedc-434e-8f90-c84ce8b5d5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:15.610975 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c61847-dedc-434e-8f90-c84ce8b5d5e4 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:15.611994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c61847-dedc-434e-8f90-c84ce8b5d5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:15.612412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c61847-dedc-434e-8f90-c84ce8b5d5e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:15.622421 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c61847-dedc-434e-8f90-c84ce8b5d5e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:15.625239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c61847-dedc-434e-8f90-c84ce8b5d5e4 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:15.627295 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d79ded-9d9e-4a62-a369-15aaa353a0b4 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:15.645330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-12c61847-dedc-434e-8f90-c84ce8b5d5e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:15.645330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:15.645330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:15.645330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:15.645330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:15.645330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:15.645330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:15.645330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:15.645330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:15.645330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:15.645330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:15.645330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:15.645330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:33:15.645330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:33:15.645330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:33:15.646428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:33:15.646428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:15.646428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:15.646428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:15.646428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:15.646428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:15.646428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:15.646428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:15.646428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:15.646428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:15.646428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:15.646428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:15.646428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:15.646428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:15.646428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:15.646428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:15.646428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:15.647686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:33:15.647686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:33:15.647686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:33:15.647686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:33:15.647686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:33:15.647686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:33:15.647686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:15.647686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:15.650268 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:15.655817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 475/952] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:33:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:15.656612 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c61847-dedc-434e-8f90-c84ce8b5d5e4 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:15.677794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:15.678102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc547e5-095d-41ea-a56d-1b6979bf0bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:15.679623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc547e5-095d-41ea-a56d-1b6979bf0bc0 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:15.680300 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc547e5-095d-41ea-a56d-1b6979bf0bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:15.680588 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc547e5-095d-41ea-a56d-1b6979bf0bc0 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:15.681272 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc547e5-095d-41ea-a56d-1b6979bf0bc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:15.684406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc547e5-095d-41ea-a56d-1b6979bf0bc0 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:15.694163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 478/953] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jun 22 11:33:15 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:33:15.788251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c61847-dedc-434e-8f90-c84ce8b5d5e4 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:15.830991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2451f9-68fd-4ca4-aee0-b92ff07c520b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:15.831707 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2451f9-68fd-4ca4-aee0-b92ff07c520b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:15.831993 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2451f9-68fd-4ca4-aee0-b92ff07c520b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:15.832272 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2451f9-68fd-4ca4-aee0-b92ff07c520b None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:15.833047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2451f9-68fd-4ca4-aee0-b92ff07c520b None admin] RBAC: Adding query filter params (name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:15.833355 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2451f9-68fd-4ca4-aee0-b92ff07c520b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:15.834806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2451f9-68fd-4ca4-aee0-b92ff07c520b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:15.857118 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 476/954] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jun 22 11:33:15 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:16.839810 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73360-fb29-452f-9304-182e7ab362d3 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:16.840326 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73360-fb29-452f-9304-182e7ab362d3 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:16.840437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73360-fb29-452f-9304-182e7ab362d3 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:16.841999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 479/955] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:16.847310 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab44d30-c2e4-4c16-ab24-08bffc22c851 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:16.847730 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab44d30-c2e4-4c16-ab24-08bffc22c851 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:16.847936 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab44d30-c2e4-4c16-ab24-08bffc22c851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:16.865550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-4ab44d30-c2e4-4c16-ab24-08bffc22c851 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:16.867096 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-4ab44d30-c2e4-4c16-ab24-08bffc22c851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:16.940473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 477/956] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:16.963494 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a73360-fb29-452f-9304-182e7ab362d3 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:17.066474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0e0ed-d95e-4cd0-8d39-c33c027baee7 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:17.066777 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0e0ed-d95e-4cd0-8d39-c33c027baee7 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:17.067026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0e0ed-d95e-4cd0-8d39-c33c027baee7 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:17.068720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 478/957] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:33:17.073170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a73360-fb29-452f-9304-182e7ab362d3 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:17.073639 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceabd4a-b48a-49c3-ba01-9a15e0d61958 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:17.073933 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceabd4a-b48a-49c3-ba01-9a15e0d61958 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:17.074219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceabd4a-b48a-49c3-ba01-9a15e0d61958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:17.080688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:17.094026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-fceabd4a-b48a-49c3-ba01-9a15e0d61958 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:17.095547 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-fceabd4a-b48a-49c3-ba01-9a15e0d61958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:17.109512 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac07b2d-2da7-448c-8239-87148279cc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:17.110183 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac07b2d-2da7-448c-8239-87148279cc50 None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:17.110461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac07b2d-2da7-448c-8239-87148279cc50 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:17.110808 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac07b2d-2da7-448c-8239-87148279cc50 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:17.111696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac07b2d-2da7-448c-8239-87148279cc50 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:17.113082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac07b2d-2da7-448c-8239-87148279cc50 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:17.135748 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 480/958] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jun 22 11:33:16 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:33:17.187530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 479/959] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:33:17.212327 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac07b2d-2da7-448c-8239-87148279cc50 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:17.263977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:17.333836 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac07b2d-2da7-448c-8239-87148279cc50 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:17.374218 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5c845-0dcc-4471-87e6-4a32b5cb8a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:17.374953 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5c845-0dcc-4471-87e6-4a32b5cb8a61 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:17.375232 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5c845-0dcc-4471-87e6-4a32b5cb8a61 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:17.375535 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5c845-0dcc-4471-87e6-4a32b5cb8a61 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:17.389504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec5c845-0dcc-4471-87e6-4a32b5cb8a61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:33:17.398833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-0ec5c845-0dcc-4471-87e6-4a32b5cb8a61 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 22 11:33:17.398833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:17.398833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:17.398833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:17.398833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:17.398833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:17.398833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:17.398833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:17.398833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:17.398833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:17.398833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:17.398833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:17.398833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:33:17.398833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:33:17.398833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:33:17.398833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:33:17.399968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:33:17.399968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:33:17.399968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:33:17.399968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:33:17.399968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:17.399968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:17.399968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:17.399968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:17.399968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:17.399968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:17.399968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:17.399968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:17.399968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:17.399968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:17.399968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:17.399968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:17.400926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:17.400926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:17.400926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:17.400926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:17.400926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:33:17.400926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:33:17.400926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:33:17.400926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:33:17.400926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:33:17.400926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:33:17.400926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 22 11:33:17.400926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:17.400926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 481/960] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jun 22 11:33:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:17.408236 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceabd4a-b48a-49c3-ba01-9a15e0d61958 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:17.515020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceabd4a-b48a-49c3-ba01-9a15e0d61958 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:17.547184 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:17.555184 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7013357-be6d-4efd-bae1-9bb0691d99c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:17.556266 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7013357-be6d-4efd-bae1-9bb0691d99c2 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:17.556810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7013357-be6d-4efd-bae1-9bb0691d99c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:17.557307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7013357-be6d-4efd-bae1-9bb0691d99c2 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:17.558337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7013357-be6d-4efd-bae1-9bb0691d99c2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:17.558855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7013357-be6d-4efd-bae1-9bb0691d99c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:17.560536 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7013357-be6d-4efd-bae1-9bb0691d99c2 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:17.570608 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 480/961] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jun 22 11:33:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:17.576438 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5c845-0dcc-4471-87e6-4a32b5cb8a61 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:17.684645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5c845-0dcc-4471-87e6-4a32b5cb8a61 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:17.686597 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:17.721373 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb04e4-6561-445d-b629-d25e925e2796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:17.722608 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb04e4-6561-445d-b629-d25e925e2796 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:17.723101 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb04e4-6561-445d-b629-d25e925e2796 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:17.723647 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb04e4-6561-445d-b629-d25e925e2796 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:17.735065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb04e4-6561-445d-b629-d25e925e2796 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:17.736925 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb04e4-6561-445d-b629-d25e925e2796 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:17.755324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-13bb04e4-6561-445d-b629-d25e925e2796 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:17.755324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:17.755324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:17.755324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:17.755324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:17.755324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:17.755324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:17.755324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:17.755324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:17.755324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:17.755324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:17.755324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:17.755324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:33:17.755324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:33:17.755324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:33:17.756328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:33:17.756328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:17.756328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:17.756328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:17.756328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:17.756328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:17.756328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:17.756328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:17.756328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:17.756328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:17.756328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:17.756328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:17.756328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:17.756328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:17.756328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:17.756328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:17.756328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:17.757232 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:33:17.757232 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:33:17.757232 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:17.757232 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:17.760438 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 482/962] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:17.766586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7013357-be6d-4efd-bae1-9bb0691d99c2 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:17.802818 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67560926-9323-4780-9c83-132282aaf25f None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:17.803474 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67560926-9323-4780-9c83-132282aaf25f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:17.803966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67560926-9323-4780-9c83-132282aaf25f None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:17.807072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 483/963] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:17.812832 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee486e01-a743-497d-844c-28947cf401a0 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:17.813317 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee486e01-a743-497d-844c-28947cf401a0 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:17.813593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee486e01-a743-497d-844c-28947cf401a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:17.832113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-ee486e01-a743-497d-844c-28947cf401a0 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:17.833288 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ee486e01-a743-497d-844c-28947cf401a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:17.897684 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7013357-be6d-4efd-bae1-9bb0691d99c2 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:17.915371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 484/964] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:17.930655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee486e01-a743-497d-844c-28947cf401a0 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:17.953234 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f953a-7d12-4742-8f4e-036b56ae349f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:17.954785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f953a-7d12-4742-8f4e-036b56ae349f None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:17.955647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f953a-7d12-4742-8f4e-036b56ae349f None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:17.956144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f953a-7d12-4742-8f4e-036b56ae349f None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:17.957301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f953a-7d12-4742-8f4e-036b56ae349f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:17.958158 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f953a-7d12-4742-8f4e-036b56ae349f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:17.959818 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f953a-7d12-4742-8f4e-036b56ae349f None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:17.984247 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 481/965] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:17.985745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f953a-7d12-4742-8f4e-036b56ae349f None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:18.035008 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:18.048339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:18.055505 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee486e01-a743-497d-844c-28947cf401a0 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:18.094413 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec3d0a8-87a6-4a5f-a3bc-cc5c7ebd060c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:18.095222 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3d0a8-87a6-4a5f-a3bc-cc5c7ebd060c None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:18.095535 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3d0a8-87a6-4a5f-a3bc-cc5c7ebd060c None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:18.096541 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3d0a8-87a6-4a5f-a3bc-cc5c7ebd060c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:18.097015 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec3d0a8-87a6-4a5f-a3bc-cc5c7ebd060c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:18.098391 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec3d0a8-87a6-4a5f-a3bc-cc5c7ebd060c None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:18.102342 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f953a-7d12-4742-8f4e-036b56ae349f None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:18.110452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-9ec3d0a8-87a6-4a5f-a3bc-cc5c7ebd060c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 22 11:33:18.110452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:18.110452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:33:18.110452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:33:18.110452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:33:18.110452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:33:18.110452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:33:18.110452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:18.110452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 22 11:33:18.110452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:18.110452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:18.110452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:18.110452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:18.110452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:18.110452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:18.110452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:18.110452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:18.111639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:18.111639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:18.111639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:18.111639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:18.111639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:18.111639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:18.111639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:33:18.111639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:33:18.111639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:33:18.111639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:33:18.111639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:18.111639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:18.111639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:18.111639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:18.111639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:18.111639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:18.112496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:18.112496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:18.112496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:18.112496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:18.112496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:18.112496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:18.112496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:18.112496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:18.112496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:18.112496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:18.112496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:33:18.112496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:18.112496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 22 11:33:18.112496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:18.112496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 485/966] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jun 22 11:33:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:18.113341 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da602c-7bfe-4441-b50b-7503e6201f75 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:18.113341 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da602c-7bfe-4441-b50b-7503e6201f75 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:18.113462 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da602c-7bfe-4441-b50b-7503e6201f75 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:18.115514 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 486/967] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:18.119975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da602c-7bfe-4441-b50b-7503e6201f75 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:18.145835 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d3d0b-985d-4e48-92ce-37185dd85afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:18.146756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d3d0b-985d-4e48-92ce-37185dd85afa None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:18.147040 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d3d0b-985d-4e48-92ce-37185dd85afa None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:18.147319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d3d0b-985d-4e48-92ce-37185dd85afa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:18.154743 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d3d0b-985d-4e48-92ce-37185dd85afa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:18.156220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d3d0b-985d-4e48-92ce-37185dd85afa None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:18.171816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-328d3d0b-985d-4e48-92ce-37185dd85afa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 22 11:33:18.171816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:18.171816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:18.171816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:18.171816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:18.171816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:18.171816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:18.171816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:18.171816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:18.171816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:18.171816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:18.171816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:18.171816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:33:18.171816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:33:18.171816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:33:18.171816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:33:18.172782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:18.172782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:18.172782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:33:18.172782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:18.172782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:33:18.172782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:18.172782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:18.172782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:18.172782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:18.172782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:18.172782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:18.172782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:18.172782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:18.172782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:18.172782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:18.172782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:18.172782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:18.172782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:18.173717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:18.173717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:18.173717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:33:18.173717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:33:18.173717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:33:18.173717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:33:18.173717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:33:18.173717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:33:18.173717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 22 11:33:18.173717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:18.173717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 482/968] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jun 22 11:33:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:18.174394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058aa63a-7595-438f-8586-31c75331be09 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:18.175326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058aa63a-7595-438f-8586-31c75331be09 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:18.175326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058aa63a-7595-438f-8586-31c75331be09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:18.191160 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-058aa63a-7595-438f-8586-31c75331be09 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:18.192387 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-058aa63a-7595-438f-8586-31c75331be09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:18.237885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da602c-7bfe-4441-b50b-7503e6201f75 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:18.266555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 483/969] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:18.268327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058aa63a-7595-438f-8586-31c75331be09 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:18.274787 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c50ed-435b-4aa2-bfa5-b80e1f1c15cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:18.275568 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c50ed-435b-4aa2-bfa5-b80e1f1c15cc None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:18.275922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c50ed-435b-4aa2-bfa5-b80e1f1c15cc None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:18.276269 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c50ed-435b-4aa2-bfa5-b80e1f1c15cc None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:18.277049 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c50ed-435b-4aa2-bfa5-b80e1f1c15cc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:18.277422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c50ed-435b-4aa2-bfa5-b80e1f1c15cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:18.278871 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c50ed-435b-4aa2-bfa5-b80e1f1c15cc None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:18.284370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 487/970] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jun 22 11:33:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:18.291461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c50ed-435b-4aa2-bfa5-b80e1f1c15cc None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:18.311127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:18.321868 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:18.386904 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058aa63a-7595-438f-8586-31c75331be09 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:18.406320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c50ed-435b-4aa2-bfa5-b80e1f1c15cc None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:18.424883 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ba173-b0c9-4580-8137-3f9f4fced1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:18.425555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ba173-b0c9-4580-8137-3f9f4fced1b7 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:18.425836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ba173-b0c9-4580-8137-3f9f4fced1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:18.426110 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ba173-b0c9-4580-8137-3f9f4fced1b7 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:18.426948 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ba173-b0c9-4580-8137-3f9f4fced1b7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:18.427256 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ba173-b0c9-4580-8137-3f9f4fced1b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:18.428693 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ba173-b0c9-4580-8137-3f9f4fced1b7 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:18.445164 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d7e3d-3f07-499a-9703-9dfeeb5adabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:18.446118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d7e3d-3f07-499a-9703-9dfeeb5adabc None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:18.446587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d7e3d-3f07-499a-9703-9dfeeb5adabc None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:18.447048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d7e3d-3f07-499a-9703-9dfeeb5adabc None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:18.447156 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 484/971] 10.222.0.36 () {56 vars in 1149 bytes} [Wed Jun 22 11:33:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:18.447876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ba173-b0c9-4580-8137-3f9f4fced1b7 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:18.448261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d7e3d-3f07-499a-9703-9dfeeb5adabc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:18.448575 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d7e3d-3f07-499a-9703-9dfeeb5adabc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:18.449950 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d7e3d-3f07-499a-9703-9dfeeb5adabc None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:18.461053 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 488/972] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jun 22 11:33:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:18.461894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d7e3d-3f07-499a-9703-9dfeeb5adabc None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:18.549703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:18.550273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:18.554061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ba173-b0c9-4580-8137-3f9f4fced1b7 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:18.576557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d7e3d-3f07-499a-9703-9dfeeb5adabc None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:18.593869 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a472f-3b5a-4359-a56f-5447b0684411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:18.594543 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a472f-3b5a-4359-a56f-5447b0684411 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:18.594828 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a472f-3b5a-4359-a56f-5447b0684411 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:18.595239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a472f-3b5a-4359-a56f-5447b0684411 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:18.595847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a472f-3b5a-4359-a56f-5447b0684411 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:18.597267 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a472f-3b5a-4359-a56f-5447b0684411 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:18.610553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.sql.core [None req-8e6a472f-3b5a-4359-a56f-5447b0684411 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 22 11:33:18.610553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 22 11:33:18.610553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [parameters: {'user_id': 'c31d64d3e28f4f81a2379fdddd40f487', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 22 11:33:18.610553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86768) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 22 11:33:18.613618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6ed99-d583-4f77-a6ed-6604a350f50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:18.614335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6ed99-d583-4f77-a6ed-6604a350f50c None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:18.614621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6ed99-d583-4f77-a6ed-6604a350f50c None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:18.614916 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6ed99-d583-4f77-a6ed-6604a350f50c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:18.618258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-8e6a472f-3b5a-4359-a56f-5447b0684411 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 22 11:33:18.618258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:18.618258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 22 11:33:18.618258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 22 11:33:18.618258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 22 11:33:18.618258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 22 11:33:18.618258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 22 11:33:18.618258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application result = self._query(query) Jun 22 11:33:18.618258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 22 11:33:18.618258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application conn.query(q) Jun 22 11:33:18.618258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 22 11:33:18.618258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 22 11:33:18.618258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 22 11:33:18.618258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application result.read() Jun 22 11:33:18.618258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 22 11:33:18.619136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 22 11:33:18.619136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 22 11:33:18.619136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 22 11:33:18.619136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 22 11:33:18.619136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 22 11:33:18.619136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 22 11:33:18.619136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 22 11:33:18.619136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 22 11:33:18.619136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:18.619136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 22 11:33:18.619136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:18.619136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:18.619136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 22 11:33:18.619136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 22 11:33:18.619136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 22 11:33:18.619136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 22 11:33:18.619136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 22 11:33:18.619136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application next(self.gen) Jun 22 11:33:18.620285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 22 11:33:18.620285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application yield resource Jun 22 11:33:18.620285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 22 11:33:18.620285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application next(self.gen) Jun 22 11:33:18.620285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 22 11:33:18.620285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self.session.rollback() Jun 22 11:33:18.620285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 22 11:33:18.620285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self.force_reraise() Jun 22 11:33:18.620285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 22 11:33:18.620285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise self.value Jun 22 11:33:18.620285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 22 11:33:18.620285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 22 11:33:18.620285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 22 11:33:18.620285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application session.commit() Jun 22 11:33:18.620285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 22 11:33:18.620285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 22 11:33:18.620285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 22 11:33:18.621171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._prepare_impl() Jun 22 11:33:18.621171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 22 11:33:18.621171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self.session.flush() Jun 22 11:33:18.621171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 22 11:33:18.621171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._flush(objects) Jun 22 11:33:18.621171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 22 11:33:18.621171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 22 11:33:18.621171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 22 11:33:18.621171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application compat.raise_( Jun 22 11:33:18.621171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 22 11:33:18.621171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception Jun 22 11:33:18.621171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 22 11:33:18.621171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application flush_context.execute() Jun 22 11:33:18.621171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 22 11:33:18.621171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rec.execute(self) Jun 22 11:33:18.621171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 22 11:33:18.621171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 22 11:33:18.622079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 22 11:33:18.622079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 22 11:33:18.622079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 22 11:33:18.622079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 22 11:33:18.622079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 22 11:33:18.622079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 22 11:33:18.622079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 22 11:33:18.622079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 22 11:33:18.622079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 22 11:33:18.622079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 22 11:33:18.622079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 22 11:33:18.622079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 22 11:33:18.622079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 22 11:33:18.622079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 22 11:33:18.622079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 22 11:33:18.622079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception Jun 22 11:33:18.622957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 22 11:33:18.622957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 22 11:33:18.622957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 22 11:33:18.622957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 22 11:33:18.622957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 22 11:33:18.622957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application result = self._query(query) Jun 22 11:33:18.622957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 22 11:33:18.622957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application conn.query(q) Jun 22 11:33:18.622957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 22 11:33:18.622957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 22 11:33:18.622957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 22 11:33:18.622957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application result.read() Jun 22 11:33:18.622957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 22 11:33:18.622957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 22 11:33:18.622957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 22 11:33:18.622957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 22 11:33:18.625708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 22 11:33:18.625708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 22 11:33:18.625708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 22 11:33:18.625708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 22 11:33:18.625708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 22 11:33:18.625708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 22 11:33:18.625708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c31d64d3e28f4f81a2379fdddd40f487', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 22 11:33:18.625708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 22 11:33:18.625708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:18.625708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:18.625708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:18.625708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:18.625708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:18.625708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:18.625708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:18.627839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:18.627839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:18.627839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:18.627839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:18.627839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:18.627839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:18.627839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:18.627839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 22 11:33:18.627839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 22 11:33:18.627839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:18.627839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:18.627839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:33:18.627839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:18.627839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:33:18.627839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:18.627839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 22 11:33:18.627839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 22 11:33:18.631123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 22 11:33:18.631123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 22 11:33:18.631123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 22 11:33:18.631123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 22 11:33:18.631123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 22 11:33:18.631123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:18.631123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 485/973] 10.222.0.36 () {60 vars in 1131 bytes} [Wed Jun 22 11:33:18 2022] POST /identity/v3/users => generated 160 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:18.631123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a472f-3b5a-4359-a56f-5447b0684411 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:18.693820 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6ed99-d583-4f77-a6ed-6604a350f50c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:18.695398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6ed99-d583-4f77-a6ed-6604a350f50c None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:18.731001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a472f-3b5a-4359-a56f-5447b0684411 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:18.767300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37526829-c960-45fd-9f24-f7c4e7c2a8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:18.768510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37526829-c960-45fd-9f24-f7c4e7c2a8e0 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:18.768790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37526829-c960-45fd-9f24-f7c4e7c2a8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:18.769066 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37526829-c960-45fd-9f24-f7c4e7c2a8e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:18.769807 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37526829-c960-45fd-9f24-f7c4e7c2a8e0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:18.771391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37526829-c960-45fd-9f24-f7c4e7c2a8e0 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:18.774258 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:18.795323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 486/974] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jun 22 11:33:18 2022] POST /identity/v3/endpoints => generated 349 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:33:18.796325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37526829-c960-45fd-9f24-f7c4e7c2a8e0 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:18.812519 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 489/975] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:33:18 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 22 11:33:18.817956 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6ed99-d583-4f77-a6ed-6604a350f50c None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:18.832019 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:18.964955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6ed99-d583-4f77-a6ed-6604a350f50c None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:18.968350 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37526829-c960-45fd-9f24-f7c4e7c2a8e0 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:19.013676 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99758e-d950-471f-91d5-26185760effe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:19.014474 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99758e-d950-471f-91d5-26185760effe None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:19.015019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99758e-d950-471f-91d5-26185760effe None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:19.015318 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99758e-d950-471f-91d5-26185760effe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:19.019072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656d56b-9743-4580-ab1c-dff59df22e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:19.020122 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656d56b-9743-4580-ab1c-dff59df22e50 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:19.020557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656d56b-9743-4580-ab1c-dff59df22e50 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:19.020983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656d56b-9743-4580-ab1c-dff59df22e50 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:19.027234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99758e-d950-471f-91d5-26185760effe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:19.028791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99758e-d950-471f-91d5-26185760effe None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:19.029972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656d56b-9743-4580-ab1c-dff59df22e50 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:19.031607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656d56b-9743-4580-ab1c-dff59df22e50 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:19.047616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-fa99758e-d950-471f-91d5-26185760effe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:19.047616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:19.047616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:19.047616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:19.047616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:19.047616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:19.047616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:19.047616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:19.047616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:19.047616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:19.047616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:19.047616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:19.047616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:33:19.047616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:33:19.047616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:33:19.048823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:33:19.048823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:19.048823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:19.048823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:19.048823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:19.048823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:19.048823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:19.048823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:19.048823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:19.048823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:19.048823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:19.048823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:19.048823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:19.048823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:19.048823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:19.048823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:19.048823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:19.049958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:33:19.049958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:33:19.049958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:19.049958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:19.049958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 490/976] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:19.050983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-c656d56b-9743-4580-ab1c-dff59df22e50 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 22 11:33:19.050983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:19.050983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:19.050983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:19.050983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:19.050983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:19.050983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:19.050983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:19.050983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:19.050983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:19.050983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:19.050983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:19.050983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:33:19.050983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:33:19.050983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:33:19.050983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:33:19.051955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:19.051955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:19.051955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:33:19.051955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:19.051955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:33:19.051955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:19.051955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:19.051955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:19.051955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:19.051955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:19.051955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:19.051955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:19.051955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:19.051955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:19.051955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:19.051955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:19.051955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:19.051955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:19.052900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:19.052900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:19.052900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:33:19.052900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:33:19.052900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:33:19.052900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:33:19.052900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:33:19.052900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:33:19.052900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 22 11:33:19.052900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:19.052900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 487/977] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:33:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:19.057918 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99758e-d950-471f-91d5-26185760effe None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:19.059948 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656d56b-9743-4580-ab1c-dff59df22e50 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:19.089095 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:19.153444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:19.176946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99758e-d950-471f-91d5-26185760effe None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:19.183204 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656d56b-9743-4580-ab1c-dff59df22e50 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:19.218447 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e15923-d142-4b78-8506-6602d42f11cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:19.219746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e15923-d142-4b78-8506-6602d42f11cd None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:19.220353 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e15923-d142-4b78-8506-6602d42f11cd None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:19.220865 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e15923-d142-4b78-8506-6602d42f11cd None admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:19.222024 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e15923-d142-4b78-8506-6602d42f11cd None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:19.222133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fd71f-79e1-4a18-ac8d-2f675d50514c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:19.222661 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e15923-d142-4b78-8506-6602d42f11cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:19.222767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fd71f-79e1-4a18-ac8d-2f675d50514c None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:19.222985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fd71f-79e1-4a18-ac8d-2f675d50514c None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:19.223268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fd71f-79e1-4a18-ac8d-2f675d50514c None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:19.224047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399fd71f-79e1-4a18-ac8d-2f675d50514c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:19.224360 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399fd71f-79e1-4a18-ac8d-2f675d50514c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:19.225296 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e15923-d142-4b78-8506-6602d42f11cd None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:19.225777 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399fd71f-79e1-4a18-ac8d-2f675d50514c None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:19.242790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 488/978] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:19.248490 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fd71f-79e1-4a18-ac8d-2f675d50514c None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:19.268606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 491/979] 10.222.0.36 () {56 vars in 1101 bytes} [Wed Jun 22 11:33:19 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:19.295658 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad57d6-ddfe-4c71-98d7-e7ddba366109 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:19.295981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad57d6-ddfe-4c71-98d7-e7ddba366109 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:19.296231 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad57d6-ddfe-4c71-98d7-e7ddba366109 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:19.297843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 492/980] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:33:19.305276 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd109932-db53-450f-acf6-78710e67df77 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:19.305941 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd109932-db53-450f-acf6-78710e67df77 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:19.306431 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd109932-db53-450f-acf6-78710e67df77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:19.331406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-cd109932-db53-450f-acf6-78710e67df77 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:19.331406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-cd109932-db53-450f-acf6-78710e67df77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:19.386119 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fd71f-79e1-4a18-ac8d-2f675d50514c None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:19.423819 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 493/981] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:19 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:33:19.435391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad4d23-c496-4710-bf4b-13e53892edd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:19.436223 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad4d23-c496-4710-bf4b-13e53892edd0 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:19.436546 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad4d23-c496-4710-bf4b-13e53892edd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:19.436822 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad4d23-c496-4710-bf4b-13e53892edd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:19.443737 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:19.446463 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd109932-db53-450f-acf6-78710e67df77 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:19.448147 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad4d23-c496-4710-bf4b-13e53892edd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:19.450293 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad4d23-c496-4710-bf4b-13e53892edd0 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:19.465890 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:19.467930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-cdad4d23-c496-4710-bf4b-13e53892edd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:19.467930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:19.467930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:19.467930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:19.467930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:19.467930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:19.467930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:19.467930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:19.467930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:19.467930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:19.467930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:19.467930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:19.467930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:33:19.467930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:33:19.467930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:33:19.469210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:33:19.469210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:19.469210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:19.469210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:19.469210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:19.469210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:19.469210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:19.469210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:19.469210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:19.469210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:19.469210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:19.469210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:19.469210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:19.469210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:19.469210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:19.469210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:19.469210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:19.470279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:33:19.470279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:33:19.470279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:33:19.470279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:33:19.470279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:33:19.470279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:33:19.470279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:19.470279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:19.470279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 489/982] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:33:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:19.475958 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad4d23-c496-4710-bf4b-13e53892edd0 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:19.571462 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd109932-db53-450f-acf6-78710e67df77 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:19.604528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad4d23-c496-4710-bf4b-13e53892edd0 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:19.614460 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea810ea3-8976-4fbd-81e3-393b7590894e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:19.615263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea810ea3-8976-4fbd-81e3-393b7590894e None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:19.615752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea810ea3-8976-4fbd-81e3-393b7590894e None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:19.616035 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea810ea3-8976-4fbd-81e3-393b7590894e None admin] PATH_INFO: `/v3/services/compute` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:19.616691 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea810ea3-8976-4fbd-81e3-393b7590894e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:19.618110 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea810ea3-8976-4fbd-81e3-393b7590894e None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:19.631227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-ea810ea3-8976-4fbd-81e3-393b7590894e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 22 11:33:19.631227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:19.631227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:33:19.631227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:33:19.631227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:33:19.631227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:33:19.631227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:33:19.631227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:19.631227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 22 11:33:19.631227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:19.631227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:19.631227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:19.631227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:19.631227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:19.631227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:19.631227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:19.631227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:19.632280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:19.632280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:19.632280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:19.632280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:19.632280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:19.632280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:19.632280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:33:19.632280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:33:19.632280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:33:19.632280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:33:19.632280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:19.632280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:19.632280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:19.632280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:19.632280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:19.632280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:19.633140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:19.633140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:19.633140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:19.633140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:19.633140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:19.633140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:19.633140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:19.633140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:19.633140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:19.633140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:19.633140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:33:19.633140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:19.633140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 22 11:33:19.633140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:19.633140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 494/983] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:33:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:19.638599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea810ea3-8976-4fbd-81e3-393b7590894e None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:19.645243 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3baef-489d-4018-87dd-20368a8af880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:19.645919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3baef-489d-4018-87dd-20368a8af880 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:19.646206 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3baef-489d-4018-87dd-20368a8af880 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:19.646486 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3baef-489d-4018-87dd-20368a8af880 None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:19.647282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3baef-489d-4018-87dd-20368a8af880 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:19.647629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3baef-489d-4018-87dd-20368a8af880 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:19.649070 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3baef-489d-4018-87dd-20368a8af880 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:19.662539 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 490/984] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:33:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:19.668306 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3baef-489d-4018-87dd-20368a8af880 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:19.724827 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:19.730837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:19.747609 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea810ea3-8976-4fbd-81e3-393b7590894e None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:19.778797 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3baef-489d-4018-87dd-20368a8af880 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:19.786661 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c516da-0792-4b99-acbb-4862c51b8eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:19.787530 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c516da-0792-4b99-acbb-4862c51b8eb6 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:19.787641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c516da-0792-4b99-acbb-4862c51b8eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:19.787910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c516da-0792-4b99-acbb-4862c51b8eb6 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:19.788967 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c516da-0792-4b99-acbb-4862c51b8eb6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:19.789277 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c516da-0792-4b99-acbb-4862c51b8eb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:19.791230 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c516da-0792-4b99-acbb-4862c51b8eb6 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:19.797717 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 495/985] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jun 22 11:33:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:19.802660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c516da-0792-4b99-acbb-4862c51b8eb6 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:19.817369 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedb093-59df-4a75-b190-50076dedb54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:19.818030 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedb093-59df-4a75-b190-50076dedb54a None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:19.818313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedb093-59df-4a75-b190-50076dedb54a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:19.818580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedb093-59df-4a75-b190-50076dedb54a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:19.819444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedb093-59df-4a75-b190-50076dedb54a None admin] RBAC: Adding query filter params (role.id=8ce71a4c1e804871b92156359277b9ee, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=955c44ffe1bd426e900c7405df44d1cf) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:19.819758 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedb093-59df-4a75-b190-50076dedb54a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:19.821175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedb093-59df-4a75-b190-50076dedb54a None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:19.835383 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 491/986] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:33:19 2022] GET /identity/v3/role_assignments?user.id=955c44ffe1bd426e900c7405df44d1cf&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=8ce71a4c1e804871b92156359277b9ee => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:19.946301 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c516da-0792-4b99-acbb-4862c51b8eb6 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:19.986421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6458fb-7c4c-46a9-8a70-50673ce43096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:19.987167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6458fb-7c4c-46a9-8a70-50673ce43096 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:19.987385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6458fb-7c4c-46a9-8a70-50673ce43096 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:19.987670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6458fb-7c4c-46a9-8a70-50673ce43096 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:19.988704 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6458fb-7c4c-46a9-8a70-50673ce43096 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:19.988704 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6458fb-7c4c-46a9-8a70-50673ce43096 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:19.990031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6458fb-7c4c-46a9-8a70-50673ce43096 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:19.995834 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 496/987] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jun 22 11:33:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:20.001601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedb093-59df-4a75-b190-50076dedb54a None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:20.122539 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedb093-59df-4a75-b190-50076dedb54a None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:20.141929 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:20.160902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b56c1d-5537-4e4c-bc40-aed27597f1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:20.161563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b56c1d-5537-4e4c-bc40-aed27597f1cc None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:20.161847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b56c1d-5537-4e4c-bc40-aed27597f1cc None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:20.162122 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b56c1d-5537-4e4c-bc40-aed27597f1cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:20.162911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b56c1d-5537-4e4c-bc40-aed27597f1cc None admin] RBAC: Adding query filter params (interface=public, service_id=fc3670269d324d50a1dec9961c7b038e, region_id=RegionOne) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:20.163223 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b56c1d-5537-4e4c-bc40-aed27597f1cc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:20.164624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b56c1d-5537-4e4c-bc40-aed27597f1cc None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:20.175001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 492/988] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jun 22 11:33:20 2022] GET /identity/v3/endpoints?service_id=fc3670269d324d50a1dec9961c7b038e&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:20.176438 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6458fb-7c4c-46a9-8a70-50673ce43096 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:20.188354 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:20.281870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6458fb-7c4c-46a9-8a70-50673ce43096 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:20.318520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b77819-8cdd-4444-822b-f87c714c3847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:20.319192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b77819-8cdd-4444-822b-f87c714c3847 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:20.319505 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b77819-8cdd-4444-822b-f87c714c3847 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:20.319780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b77819-8cdd-4444-822b-f87c714c3847 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:20.320456 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b77819-8cdd-4444-822b-f87c714c3847 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:20.323545 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b77819-8cdd-4444-822b-f87c714c3847 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:20.329388 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 497/989] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jun 22 11:33:20 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:33:21.021745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649632b6-ae38-4a66-992c-1a22928d445e None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:21.022360 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649632b6-ae38-4a66-992c-1a22928d445e None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:21.022360 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649632b6-ae38-4a66-992c-1a22928d445e None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:21.023749 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 493/990] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:21.028932 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a3b63-7daa-4501-b715-0088a2545542 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:21.029347 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a3b63-7daa-4501-b715-0088a2545542 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:21.029523 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a3b63-7daa-4501-b715-0088a2545542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:21.046051 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-5e7a3b63-7daa-4501-b715-0088a2545542 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:21.047239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5e7a3b63-7daa-4501-b715-0088a2545542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:21.116783 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 498/991] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:33:21.140038 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649632b6-ae38-4a66-992c-1a22928d445e None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:21.249538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649632b6-ae38-4a66-992c-1a22928d445e None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:21.285394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b9b1f-be1e-47a9-a2d0-e84948211fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:21.286208 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b9b1f-be1e-47a9-a2d0-e84948211fcc None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:21.286494 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b9b1f-be1e-47a9-a2d0-e84948211fcc None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:21.286771 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b9b1f-be1e-47a9-a2d0-e84948211fcc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:21.287448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b9b1f-be1e-47a9-a2d0-e84948211fcc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:21.288819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b9b1f-be1e-47a9-a2d0-e84948211fcc None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:21.301683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-f35b9b1f-be1e-47a9-a2d0-e84948211fcc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 22 11:33:21.301683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:21.301683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:33:21.301683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:33:21.301683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:33:21.301683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:33:21.301683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:33:21.301683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:21.301683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 22 11:33:21.301683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:21.301683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:21.301683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:21.301683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:21.301683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:21.301683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:21.301683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:21.301683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:21.302819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:21.302819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:21.302819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:21.302819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:21.302819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:21.302819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:21.302819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:33:21.302819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:33:21.302819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:33:21.302819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:33:21.302819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:21.302819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:21.302819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:21.302819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:21.302819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:21.302819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:21.303692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:21.303692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:21.303692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:21.303692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:21.303692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:21.303692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:21.303692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:21.303692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:21.303692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:21.303692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:21.303692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:33:21.303692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:21.303692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 22 11:33:21.303692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:21.305035 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 494/992] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jun 22 11:33:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:21.309324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a3b63-7daa-4501-b715-0088a2545542 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:21.311729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:21.442753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a3b63-7daa-4501-b715-0088a2545542 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:21.478993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a9961-17ae-4ef5-b1f1-e7d8a3c94376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:21.479700 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a9961-17ae-4ef5-b1f1-e7d8a3c94376 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:21.480081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a9961-17ae-4ef5-b1f1-e7d8a3c94376 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:21.480306 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a9961-17ae-4ef5-b1f1-e7d8a3c94376 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:21.480943 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a9961-17ae-4ef5-b1f1-e7d8a3c94376 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:21.481246 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a9961-17ae-4ef5-b1f1-e7d8a3c94376 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:21.482612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a9961-17ae-4ef5-b1f1-e7d8a3c94376 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:21.487807 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 499/993] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jun 22 11:33:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:21.493459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b9b1f-be1e-47a9-a2d0-e84948211fcc None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:21.548551 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:21.605440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b9b1f-be1e-47a9-a2d0-e84948211fcc None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:21.641539 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f0225-a3d8-492f-b700-e26a0ea77907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:21.642292 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f0225-a3d8-492f-b700-e26a0ea77907 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:21.642602 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f0225-a3d8-492f-b700-e26a0ea77907 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:21.642880 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f0225-a3d8-492f-b700-e26a0ea77907 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:21.643609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f0225-a3d8-492f-b700-e26a0ea77907 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:21.643919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f0225-a3d8-492f-b700-e26a0ea77907 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:21.645277 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f0225-a3d8-492f-b700-e26a0ea77907 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:21.654928 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 495/994] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jun 22 11:33:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:21.656529 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a9961-17ae-4ef5-b1f1-e7d8a3c94376 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:21.735649 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:21.785376 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a9961-17ae-4ef5-b1f1-e7d8a3c94376 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:21.829348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51786aea-5e69-446e-8a76-631a88ca5296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:21.829970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51786aea-5e69-446e-8a76-631a88ca5296 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:21.830246 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51786aea-5e69-446e-8a76-631a88ca5296 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:21.830595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51786aea-5e69-446e-8a76-631a88ca5296 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:21.831429 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51786aea-5e69-446e-8a76-631a88ca5296 None admin] RBAC: Adding query filter params (interface=admin, service_id=574162db2ef04550a1a686abcb6db4ef, region_id=RegionOne) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:21.831739 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51786aea-5e69-446e-8a76-631a88ca5296 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:21.833123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51786aea-5e69-446e-8a76-631a88ca5296 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:21.843554 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 500/995] 10.222.0.36 () {56 vars in 1249 bytes} [Wed Jun 22 11:33:21 2022] GET /identity/v3/endpoints?service_id=574162db2ef04550a1a686abcb6db4ef&interface=admin®ion_id=RegionOne => generated 193 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:21.845199 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f0225-a3d8-492f-b700-e26a0ea77907 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:21.912682 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020022f5-a8c2-4c78-aef8-36a33b234c4f None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:21.912975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020022f5-a8c2-4c78-aef8-36a33b234c4f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:21.913252 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020022f5-a8c2-4c78-aef8-36a33b234c4f None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:21.914780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 501/996] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:21.919547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415780ff-8b6e-4e11-ab4a-6231d80ecb2c None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:21.919836 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415780ff-8b6e-4e11-ab4a-6231d80ecb2c None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:21.920111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415780ff-8b6e-4e11-ab4a-6231d80ecb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:21.940717 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-415780ff-8b6e-4e11-ab4a-6231d80ecb2c None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:21.941925 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-415780ff-8b6e-4e11-ab4a-6231d80ecb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:21.965555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f0225-a3d8-492f-b700-e26a0ea77907 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:21.978721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:22.012893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43705bc-17d3-4022-8457-76aff63b6b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:22.013826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43705bc-17d3-4022-8457-76aff63b6b68 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:22.014108 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43705bc-17d3-4022-8457-76aff63b6b68 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:22.014396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43705bc-17d3-4022-8457-76aff63b6b68 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:22.015368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43705bc-17d3-4022-8457-76aff63b6b68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:22.016771 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43705bc-17d3-4022-8457-76aff63b6b68 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:22.023697 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 496/997] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jun 22 11:33:21 2022] GET /identity/v3/services => generated 1727 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:33:22.027296 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 502/998] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:21 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:33:22.051043 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43705bc-17d3-4022-8457-76aff63b6b68 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:22.167277 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43705bc-17d3-4022-8457-76aff63b6b68 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:22.205745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78397b65-7957-4555-b7ab-67194d9e4cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:22.206541 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78397b65-7957-4555-b7ab-67194d9e4cdf None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:22.206870 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78397b65-7957-4555-b7ab-67194d9e4cdf None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:22.207099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78397b65-7957-4555-b7ab-67194d9e4cdf None admin] PATH_INFO: `/v3/services/image` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:22.207844 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78397b65-7957-4555-b7ab-67194d9e4cdf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:22.209215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78397b65-7957-4555-b7ab-67194d9e4cdf None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:22.221656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-78397b65-7957-4555-b7ab-67194d9e4cdf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 22 11:33:22.221656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:22.221656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:33:22.221656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:33:22.221656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:33:22.221656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:33:22.221656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:33:22.221656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:22.221656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 22 11:33:22.221656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:22.221656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:22.221656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:22.221656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:22.221656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:22.221656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:22.221656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:22.221656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:22.222775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:22.222775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:22.222775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:22.222775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:22.222775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:22.222775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:22.222775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:33:22.222775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:33:22.222775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:33:22.222775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:33:22.222775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:22.222775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:22.222775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:22.222775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:22.222775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:22.222775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:22.224323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:22.224323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:22.224323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:22.224323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:22.224323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:22.224323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:22.224323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:22.224323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:22.224323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:22.224323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:22.224323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:33:22.224323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:22.224323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 22 11:33:22.224323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:22.224323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 497/999] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Jun 22 11:33:22 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:22.228945 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415780ff-8b6e-4e11-ab4a-6231d80ecb2c None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:22.289806 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:22.333016 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415780ff-8b6e-4e11-ab4a-6231d80ecb2c None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:22.369133 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42b520-8d4d-4802-850a-87296683c6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:22.369806 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42b520-8d4d-4802-850a-87296683c6b5 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:22.370204 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42b520-8d4d-4802-850a-87296683c6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:22.370394 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42b520-8d4d-4802-850a-87296683c6b5 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:22.371080 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42b520-8d4d-4802-850a-87296683c6b5 None admin] RBAC: Adding query filter params (name=image) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:22.371414 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42b520-8d4d-4802-850a-87296683c6b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:22.372799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42b520-8d4d-4802-850a-87296683c6b5 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:22.378818 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 503/1000] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:22.384309 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78397b65-7957-4555-b7ab-67194d9e4cdf None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:22.396362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac60ff-9582-4094-b3f4-3844e1191531 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:22.396652 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac60ff-9582-4094-b3f4-3844e1191531 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:22.396960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac60ff-9582-4094-b3f4-3844e1191531 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:22.398530 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 504/1001] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:33:22.404361 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472261b-f939-4142-b11e-7130d568ae2e None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:22.404361 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472261b-f939-4142-b11e-7130d568ae2e None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:22.404598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472261b-f939-4142-b11e-7130d568ae2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:22.421961 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-5472261b-f939-4142-b11e-7130d568ae2e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:22.423128 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5472261b-f939-4142-b11e-7130d568ae2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:22.428672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:22.507296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78397b65-7957-4555-b7ab-67194d9e4cdf None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:22.511131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 505/1002] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:22 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:22.536042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472261b-f939-4142-b11e-7130d568ae2e None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:22.554095 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332fbb6b-9fc3-4eae-a616-33a360b900d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:22.558435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fbb6b-9fc3-4eae-a616-33a360b900d2 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:22.558435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fbb6b-9fc3-4eae-a616-33a360b900d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:22.558435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fbb6b-9fc3-4eae-a616-33a360b900d2 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:22.558435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332fbb6b-9fc3-4eae-a616-33a360b900d2 None admin] RBAC: Adding query filter params (type=image) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:22.558435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332fbb6b-9fc3-4eae-a616-33a360b900d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:22.558435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332fbb6b-9fc3-4eae-a616-33a360b900d2 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:22.563155 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 498/1003] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:22 2022] GET /identity/v3/services?type=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:22.633126 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:22.693457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472261b-f939-4142-b11e-7130d568ae2e None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:22.733083 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3453f-ea72-4748-9b8b-21b4104f06d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:22.733892 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3453f-ea72-4748-9b8b-21b4104f06d2 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:22.734179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3453f-ea72-4748-9b8b-21b4104f06d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:22.734457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3453f-ea72-4748-9b8b-21b4104f06d2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:22.735115 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3453f-ea72-4748-9b8b-21b4104f06d2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:22.737150 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3453f-ea72-4748-9b8b-21b4104f06d2 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:22.749582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-d9e3453f-ea72-4748-9b8b-21b4104f06d2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 22 11:33:22.749582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:22.749582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:33:22.749582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:33:22.749582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:33:22.749582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:33:22.749582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:33:22.749582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:22.749582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 22 11:33:22.749582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:22.749582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:22.749582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:22.749582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:22.749582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:22.749582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:22.749582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:22.749582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:22.750632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:22.750632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:22.750632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:22.750632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:22.750632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:22.750632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:22.750632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:33:22.750632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:33:22.750632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:33:22.750632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:33:22.750632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:22.750632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:22.750632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:22.750632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:22.750632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:22.750632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:22.751660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:22.751660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:22.751660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:22.751660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:22.751660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:22.751660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:22.751660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:22.751660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:22.751660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:22.751660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:22.751660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:33:22.751660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:22.751660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 22 11:33:22.751660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:22.751660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 506/1004] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:33:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:22.758301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332fbb6b-9fc3-4eae-a616-33a360b900d2 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:22.863592 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332fbb6b-9fc3-4eae-a616-33a360b900d2 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:22.865126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:22.898279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a75ed-ef87-4aa6-b832-e8fed2de7d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:22.898934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a75ed-ef87-4aa6-b832-e8fed2de7d45 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:22.899212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a75ed-ef87-4aa6-b832-e8fed2de7d45 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:22.899516 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a75ed-ef87-4aa6-b832-e8fed2de7d45 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:22.900217 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a75ed-ef87-4aa6-b832-e8fed2de7d45 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:22.900516 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a75ed-ef87-4aa6-b832-e8fed2de7d45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:22.901882 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a75ed-ef87-4aa6-b832-e8fed2de7d45 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:22.912104 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 499/1005] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jun 22 11:33:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:22.913724 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3453f-ea72-4748-9b8b-21b4104f06d2 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:23.023242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3453f-ea72-4748-9b8b-21b4104f06d2 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:23.040455 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:23.058032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01530b25-faf4-4f3a-b165-13ea37d09ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:23.058761 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01530b25-faf4-4f3a-b165-13ea37d09ba5 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:23.059039 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01530b25-faf4-4f3a-b165-13ea37d09ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:23.059310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01530b25-faf4-4f3a-b165-13ea37d09ba5 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:23.060066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01530b25-faf4-4f3a-b165-13ea37d09ba5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:23.060449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01530b25-faf4-4f3a-b165-13ea37d09ba5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:23.061740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01530b25-faf4-4f3a-b165-13ea37d09ba5 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:23.072708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 507/1006] 10.222.0.36 () {56 vars in 1112 bytes} [Wed Jun 22 11:33:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:23.074427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a75ed-ef87-4aa6-b832-e8fed2de7d45 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:23.184937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a75ed-ef87-4aa6-b832-e8fed2de7d45 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:23.222594 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5cd7a6-d5b3-4ec5-96cb-17f47a220523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:23.223159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cd7a6-d5b3-4ec5-96cb-17f47a220523 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:23.223479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cd7a6-d5b3-4ec5-96cb-17f47a220523 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:23.223756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5cd7a6-d5b3-4ec5-96cb-17f47a220523 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:23.224499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5cd7a6-d5b3-4ec5-96cb-17f47a220523 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:23.225869 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5cd7a6-d5b3-4ec5-96cb-17f47a220523 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:23.246195 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 500/1007] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jun 22 11:33:23 2022] POST /identity/v3/endpoints => generated 334 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:33:24.103305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc7daf-e0b4-4909-8020-3d0612bdd047 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:24.103846 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc7daf-e0b4-4909-8020-3d0612bdd047 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:24.103958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc7daf-e0b4-4909-8020-3d0612bdd047 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:24.105303 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 508/1008] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:24.113798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21edc61-b36a-49b1-bf2d-1d35c8d8ec46 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:24.114288 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21edc61-b36a-49b1-bf2d-1d35c8d8ec46 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:24.114364 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21edc61-b36a-49b1-bf2d-1d35c8d8ec46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:24.131843 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-b21edc61-b36a-49b1-bf2d-1d35c8d8ec46 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:24.133365 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b21edc61-b36a-49b1-bf2d-1d35c8d8ec46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:24.150245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:24.208575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 501/1009] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:24 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:24.231682 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc7daf-e0b4-4909-8020-3d0612bdd047 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:24.338516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc7daf-e0b4-4909-8020-3d0612bdd047 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:24.372964 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906070d-d000-4120-bd64-7940725ce001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:24.373766 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906070d-d000-4120-bd64-7940725ce001 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:24.374048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906070d-d000-4120-bd64-7940725ce001 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:24.374324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906070d-d000-4120-bd64-7940725ce001 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:24.374971 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906070d-d000-4120-bd64-7940725ce001 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:24.376382 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906070d-d000-4120-bd64-7940725ce001 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:24.388451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-3906070d-d000-4120-bd64-7940725ce001 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 22 11:33:24.388451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:24.388451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:33:24.388451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:33:24.388451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:33:24.388451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:33:24.388451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:33:24.388451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:24.388451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 22 11:33:24.388451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:24.388451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:24.388451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:24.388451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:24.388451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:24.388451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:24.388451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:24.388451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:24.389458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:24.389458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:24.389458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:24.389458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:24.389458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:24.389458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:24.389458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:33:24.389458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:33:24.389458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:33:24.389458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:33:24.389458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:24.389458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:24.389458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:24.389458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:24.389458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:24.389458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:24.390567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:24.390567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:24.390567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:24.390567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:24.390567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:24.390567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:24.390567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:24.390567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:24.390567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:24.390567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:24.390567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:33:24.390567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:24.390567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 22 11:33:24.390567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:24.392280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 509/1010] 10.222.0.36 () {56 vars in 1126 bytes} [Wed Jun 22 11:33:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:24.396092 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21edc61-b36a-49b1-bf2d-1d35c8d8ec46 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:24.500192 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21edc61-b36a-49b1-bf2d-1d35c8d8ec46 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:24.536536 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86893c-1376-40ec-a938-c60c159cbe8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:24.537275 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86893c-1376-40ec-a938-c60c159cbe8d None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:24.537564 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86893c-1376-40ec-a938-c60c159cbe8d None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:24.537830 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86893c-1376-40ec-a938-c60c159cbe8d None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:24.538538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86893c-1376-40ec-a938-c60c159cbe8d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:24.538840 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86893c-1376-40ec-a938-c60c159cbe8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:24.540249 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86893c-1376-40ec-a938-c60c159cbe8d None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:24.550630 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 502/1011] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jun 22 11:33:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:24.551851 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906070d-d000-4120-bd64-7940725ce001 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:24.600362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:24.661448 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906070d-d000-4120-bd64-7940725ce001 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:24.676671 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69a138-1eb8-4daa-b2dd-e6344656e476 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:24.676977 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69a138-1eb8-4daa-b2dd-e6344656e476 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:24.677230 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69a138-1eb8-4daa-b2dd-e6344656e476 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:24.678786 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 503/1012] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:33:24.683387 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222ea4e-a5ce-41eb-a078-7316abc23137 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:24.683640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222ea4e-a5ce-41eb-a078-7316abc23137 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:24.683923 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222ea4e-a5ce-41eb-a078-7316abc23137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:24.698030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5e735-06b2-45dd-bb3d-53c5e78e8876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:24.698701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5e735-06b2-45dd-bb3d-53c5e78e8876 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:24.698987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5e735-06b2-45dd-bb3d-53c5e78e8876 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:24.699266 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5e735-06b2-45dd-bb3d-53c5e78e8876 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:24.700012 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5e735-06b2-45dd-bb3d-53c5e78e8876 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:24.700319 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5e735-06b2-45dd-bb3d-53c5e78e8876 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:24.701582 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-f222ea4e-a5ce-41eb-a078-7316abc23137 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:24.701695 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5e735-06b2-45dd-bb3d-53c5e78e8876 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:24.702778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f222ea4e-a5ce-41eb-a078-7316abc23137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:24.712326 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 510/1013] 10.222.0.36 () {56 vars in 1122 bytes} [Wed Jun 22 11:33:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:24.713133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:24.714106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5e735-06b2-45dd-bb3d-53c5e78e8876 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:24.789464 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 504/1014] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:24 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:33:24.811763 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222ea4e-a5ce-41eb-a078-7316abc23137 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:24.819187 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5e735-06b2-45dd-bb3d-53c5e78e8876 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:24.831125 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:24.861250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073edbb-a095-4943-8dfc-5833f75a6bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:24.862235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073edbb-a095-4943-8dfc-5833f75a6bf1 None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:24.862667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073edbb-a095-4943-8dfc-5833f75a6bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:24.863079 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073edbb-a095-4943-8dfc-5833f75a6bf1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:24.864044 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073edbb-a095-4943-8dfc-5833f75a6bf1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:24.865420 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073edbb-a095-4943-8dfc-5833f75a6bf1 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:24.881549 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 511/1015] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jun 22 11:33:24 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:33:24.933650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222ea4e-a5ce-41eb-a078-7316abc23137 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:24.981077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea95916-2017-49cc-8282-0a80a1b58627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:24.981920 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea95916-2017-49cc-8282-0a80a1b58627 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:24.982300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea95916-2017-49cc-8282-0a80a1b58627 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:24.982675 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea95916-2017-49cc-8282-0a80a1b58627 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:24.983540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea95916-2017-49cc-8282-0a80a1b58627 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:24.985090 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea95916-2017-49cc-8282-0a80a1b58627 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:24.995229 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:25.007542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 505/1016] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jun 22 11:33:24 2022] POST /identity/v3/services => generated 242 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:33:25.573111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95949219-a0e8-4677-985c-6181035d4521 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:25.573662 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95949219-a0e8-4677-985c-6181035d4521 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:25.573662 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95949219-a0e8-4677-985c-6181035d4521 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:25.575156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 512/1017] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:25.580877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44baff1-6a8a-4c10-bd0f-603af06405b5 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:25.581265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44baff1-6a8a-4c10-bd0f-603af06405b5 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:25.581488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44baff1-6a8a-4c10-bd0f-603af06405b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:25.598162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c44baff1-6a8a-4c10-bd0f-603af06405b5 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:25.599337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c44baff1-6a8a-4c10-bd0f-603af06405b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:25.673376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 506/1018] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:25.697508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95949219-a0e8-4677-985c-6181035d4521 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:25.807115 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95949219-a0e8-4677-985c-6181035d4521 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:25.839875 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73568b-c2e1-4e17-942e-cda477ff5be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:25.840603 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73568b-c2e1-4e17-942e-cda477ff5be2 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:25.840883 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73568b-c2e1-4e17-942e-cda477ff5be2 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:25.841155 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73568b-c2e1-4e17-942e-cda477ff5be2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:25.852166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73568b-c2e1-4e17-942e-cda477ff5be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:33:25.860106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-6d73568b-c2e1-4e17-942e-cda477ff5be2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 22 11:33:25.860106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:25.860106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:25.860106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:25.860106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:25.860106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:25.860106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:25.860106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:25.860106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:25.860106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:25.860106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:25.860106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:25.860106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:33:25.860106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:33:25.860106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:33:25.860106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:33:25.861680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:33:25.861680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:33:25.861680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:33:25.861680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:33:25.861680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:25.861680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:25.861680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:25.861680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:25.861680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:25.861680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:25.861680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:25.861680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:25.861680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:25.861680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:25.861680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:25.861680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:25.862558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:25.862558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:25.862558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:25.862558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:25.862558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:33:25.862558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:33:25.862558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:33:25.862558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:33:25.862558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:33:25.862558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:33:25.862558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 22 11:33:25.862558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:25.867061 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 513/1019] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jun 22 11:33:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:25.868511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44baff1-6a8a-4c10-bd0f-603af06405b5 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:25.979601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44baff1-6a8a-4c10-bd0f-603af06405b5 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:26.006600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:26.018566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b5539-a7d6-488e-a2e0-326fde9d1d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:26.019236 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b5539-a7d6-488e-a2e0-326fde9d1d78 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:26.019538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b5539-a7d6-488e-a2e0-326fde9d1d78 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:26.019815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b5539-a7d6-488e-a2e0-326fde9d1d78 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:26.020568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b5539-a7d6-488e-a2e0-326fde9d1d78 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:26.020868 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b5539-a7d6-488e-a2e0-326fde9d1d78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:26.022250 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b5539-a7d6-488e-a2e0-326fde9d1d78 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:26.037548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 507/1020] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jun 22 11:33:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:26.038804 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73568b-c2e1-4e17-942e-cda477ff5be2 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:26.072816 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:26.156240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73568b-c2e1-4e17-942e-cda477ff5be2 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:26.196024 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25183fb7-9ec8-422a-ad3d-3a974fda604f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:26.197136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25183fb7-9ec8-422a-ad3d-3a974fda604f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:26.197419 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25183fb7-9ec8-422a-ad3d-3a974fda604f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:26.197696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25183fb7-9ec8-422a-ad3d-3a974fda604f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:26.210183 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25183fb7-9ec8-422a-ad3d-3a974fda604f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:26.211927 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25183fb7-9ec8-422a-ad3d-3a974fda604f None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:26.230648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-25183fb7-9ec8-422a-ad3d-3a974fda604f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:26.230648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:26.230648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:26.230648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:26.230648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:26.230648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:26.230648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:26.230648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:26.230648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:26.230648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:26.230648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:26.230648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:26.230648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:33:26.230648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:33:26.230648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:33:26.231667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:33:26.231667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:26.231667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:26.231667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:26.231667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:26.231667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:26.231667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:26.231667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:26.231667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:26.231667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:26.231667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:26.231667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:26.231667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:26.231667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:26.231667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:26.231667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:26.231667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:26.232603 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:33:26.232603 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:33:26.232603 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:26.232603 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:26.237428 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 514/1021] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:26.242989 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b5539-a7d6-488e-a2e0-326fde9d1d78 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:26.348234 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b5539-a7d6-488e-a2e0-326fde9d1d78 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:26.382151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89369bed-e1bc-44b1-b3c8-3d54c821e0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:26.382821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89369bed-e1bc-44b1-b3c8-3d54c821e0af None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:26.383226 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89369bed-e1bc-44b1-b3c8-3d54c821e0af None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:26.383448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89369bed-e1bc-44b1-b3c8-3d54c821e0af None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:26.384169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89369bed-e1bc-44b1-b3c8-3d54c821e0af None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:26.384477 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89369bed-e1bc-44b1-b3c8-3d54c821e0af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:26.385845 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89369bed-e1bc-44b1-b3c8-3d54c821e0af None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:26.404470 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 508/1022] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:26.405645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25183fb7-9ec8-422a-ad3d-3a974fda604f None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:26.496333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:26.518633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25183fb7-9ec8-422a-ad3d-3a974fda604f None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:26.556986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93936af-ab80-477e-b066-c2528ee20ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:26.557880 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93936af-ab80-477e-b066-c2528ee20ed0 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:26.558231 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93936af-ab80-477e-b066-c2528ee20ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:26.558570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93936af-ab80-477e-b066-c2528ee20ed0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:26.567199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93936af-ab80-477e-b066-c2528ee20ed0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:26.568761 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93936af-ab80-477e-b066-c2528ee20ed0 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:26.584033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-f93936af-ab80-477e-b066-c2528ee20ed0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 22 11:33:26.584033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:26.584033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:26.584033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:26.584033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:26.584033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:26.584033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:26.584033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:26.584033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:26.584033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:26.584033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:26.584033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:26.584033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:33:26.584033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:33:26.584033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:33:26.584033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:33:26.585726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:26.585726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:26.585726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:33:26.585726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:26.585726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:33:26.585726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:26.585726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:26.585726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:26.585726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:26.585726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:26.585726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:26.585726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:26.585726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:26.585726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:26.585726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:26.585726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:26.585726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:26.585726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:26.587200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:26.587200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:26.587200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:33:26.587200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:33:26.587200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:33:26.587200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:33:26.587200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:33:26.587200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:33:26.587200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 22 11:33:26.587200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:26.587200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 515/1023] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jun 22 11:33:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:26.590922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89369bed-e1bc-44b1-b3c8-3d54c821e0af None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:26.641090 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:26.697484 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89369bed-e1bc-44b1-b3c8-3d54c821e0af None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:26.733539 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1facff9-e585-4e45-9e29-77e808cbd1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:26.734212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1facff9-e585-4e45-9e29-77e808cbd1b8 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:26.734497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1facff9-e585-4e45-9e29-77e808cbd1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:26.734772 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1facff9-e585-4e45-9e29-77e808cbd1b8 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:26.735644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1facff9-e585-4e45-9e29-77e808cbd1b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:26.735984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1facff9-e585-4e45-9e29-77e808cbd1b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:26.737414 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1facff9-e585-4e45-9e29-77e808cbd1b8 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:26.760389 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 509/1024] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jun 22 11:33:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:26.761930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93936af-ab80-477e-b066-c2528ee20ed0 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:26.867379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93936af-ab80-477e-b066-c2528ee20ed0 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:26.904077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b033bf-38e9-4c75-82fa-d965415ca882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:26.904729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b033bf-38e9-4c75-82fa-d965415ca882 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:26.905012 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b033bf-38e9-4c75-82fa-d965415ca882 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:26.905290 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b033bf-38e9-4c75-82fa-d965415ca882 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:26.917888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:26.984548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b033bf-38e9-4c75-82fa-d965415ca882 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:26.986532 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b033bf-38e9-4c75-82fa-d965415ca882 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:27.019342 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4bd5d-9032-45eb-ab3e-30c2b1fe4747 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:27.019646 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4bd5d-9032-45eb-ab3e-30c2b1fe4747 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:27.019843 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4bd5d-9032-45eb-ab3e-30c2b1fe4747 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:27.021473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 510/1025] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:27.027649 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2371821a-0f80-459a-8db0-0afa8e4f60f2 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:27.027943 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2371821a-0f80-459a-8db0-0afa8e4f60f2 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:27.028225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2371821a-0f80-459a-8db0-0afa8e4f60f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:27.045226 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-2371821a-0f80-459a-8db0-0afa8e4f60f2 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:27.046415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2371821a-0f80-459a-8db0-0afa8e4f60f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:27.093595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 516/1026] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:33:26 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 22 11:33:27.100152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b033bf-38e9-4c75-82fa-d965415ca882 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:27.131225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 511/1027] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:33:27.136863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f4e5c-99ce-42b9-bbbc-50cc15371a20 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:27.137202 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f4e5c-99ce-42b9-bbbc-50cc15371a20 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:27.137611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f4e5c-99ce-42b9-bbbc-50cc15371a20 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:27.139393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 512/1028] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:27.145422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f45c7-6b08-4ada-b555-95dd834c4bf4 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:27.145708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f45c7-6b08-4ada-b555-95dd834c4bf4 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:27.145988 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f45c7-6b08-4ada-b555-95dd834c4bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:27.151516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:27.164137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-705f45c7-6b08-4ada-b555-95dd834c4bf4 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:27.165353 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-705f45c7-6b08-4ada-b555-95dd834c4bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:27.197632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:27.215172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b033bf-38e9-4c75-82fa-d965415ca882 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:27.242998 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 513/1029] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:33:27.243973 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f45c7-6b08-4ada-b555-95dd834c4bf4 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:27.263623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296856e4-db1c-486a-b88f-e22846f272e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:27.263852 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296856e4-db1c-486a-b88f-e22846f272e0 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:27.264154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296856e4-db1c-486a-b88f-e22846f272e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:27.264430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296856e4-db1c-486a-b88f-e22846f272e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:27.276162 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296856e4-db1c-486a-b88f-e22846f272e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:27.277551 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296856e4-db1c-486a-b88f-e22846f272e0 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:27.296215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-296856e4-db1c-486a-b88f-e22846f272e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:27.296215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:27.296215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:27.296215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:27.296215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:27.296215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:27.296215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:27.296215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:27.296215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:27.296215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:27.296215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:27.296215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:27.296215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:33:27.296215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:33:27.296215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:33:27.298234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:33:27.298234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:27.298234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:27.298234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:27.298234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:27.298234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:27.298234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:27.298234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:27.298234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:27.298234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:27.298234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:27.298234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:27.298234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:27.298234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:27.298234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:27.298234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:27.298234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:27.300732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:33:27.300732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:33:27.300732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:27.300732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:27.300732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 517/1030] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:27.300732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296856e4-db1c-486a-b88f-e22846f272e0 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:27.357206 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f45c7-6b08-4ada-b555-95dd834c4bf4 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:27.394897 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c1e2f-482e-412b-8c15-d116869ff8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:27.395719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c1e2f-482e-412b-8c15-d116869ff8da None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:27.395996 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c1e2f-482e-412b-8c15-d116869ff8da None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:27.396270 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c1e2f-482e-412b-8c15-d116869ff8da None admin] PATH_INFO: `/v3/domains/default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:27.397459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c1e2f-482e-412b-8c15-d116869ff8da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:27.398702 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:27.398888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c1e2f-482e-412b-8c15-d116869ff8da None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:27.400774 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 514/1031] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:27.401552 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c1e2f-482e-412b-8c15-d116869ff8da None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:27.411565 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296856e4-db1c-486a-b88f-e22846f272e0 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:27.443838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:27.453795 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd463aa3-d9a4-4eda-b85b-e58e321d5996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:27.454627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd463aa3-d9a4-4eda-b85b-e58e321d5996 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:27.454924 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd463aa3-d9a4-4eda-b85b-e58e321d5996 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:27.455199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd463aa3-d9a4-4eda-b85b-e58e321d5996 None admin] PATH_INFO: `/v3/services/image` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:27.455894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd463aa3-d9a4-4eda-b85b-e58e321d5996 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:27.457511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd463aa3-d9a4-4eda-b85b-e58e321d5996 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:27.469745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-fd463aa3-d9a4-4eda-b85b-e58e321d5996 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 22 11:33:27.469745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:27.469745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:33:27.469745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:33:27.469745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:33:27.469745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:33:27.469745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:33:27.469745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:27.469745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 22 11:33:27.469745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:27.469745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:27.469745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:27.469745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:27.469745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:27.469745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:27.469745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:27.469745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:27.471621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:27.471621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:27.471621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:27.471621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:27.471621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:27.471621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:27.471621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:33:27.471621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:33:27.471621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:33:27.471621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:33:27.471621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:27.471621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:27.471621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:27.471621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:27.471621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:27.471621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:27.472909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:27.472909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:27.472909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:27.472909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:27.472909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:27.472909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:27.472909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:27.472909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:27.472909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:27.472909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:27.472909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:33:27.472909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:27.472909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 22 11:33:27.472909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:27.472909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 518/1032] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Jun 22 11:33:27 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:27.473765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd463aa3-d9a4-4eda-b85b-e58e321d5996 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:27.543086 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c1e2f-482e-412b-8c15-d116869ff8da None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:27.584209 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf60f27-e386-40c2-967c-a1ab957e2c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:27.585576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf60f27-e386-40c2-967c-a1ab957e2c11 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:27.585576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf60f27-e386-40c2-967c-a1ab957e2c11 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:27.585576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf60f27-e386-40c2-967c-a1ab957e2c11 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:27.586150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf60f27-e386-40c2-967c-a1ab957e2c11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:27.586454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf60f27-e386-40c2-967c-a1ab957e2c11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:27.588107 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf60f27-e386-40c2-967c-a1ab957e2c11 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:27.600678 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 515/1033] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:27.601291 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd463aa3-d9a4-4eda-b85b-e58e321d5996 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:27.601684 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf60f27-e386-40c2-967c-a1ab957e2c11 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:27.640721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afc7f2-7448-4ac7-8bdb-8d5400ec98f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:27.641421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afc7f2-7448-4ac7-8bdb-8d5400ec98f4 None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:27.641702 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afc7f2-7448-4ac7-8bdb-8d5400ec98f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:27.641977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afc7f2-7448-4ac7-8bdb-8d5400ec98f4 None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:27.643228 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afc7f2-7448-4ac7-8bdb-8d5400ec98f4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:27.644730 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afc7f2-7448-4ac7-8bdb-8d5400ec98f4 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:27.672705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 519/1034] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jun 22 11:33:27 2022] POST /identity/v3/projects => generated 311 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:33:27.674395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afc7f2-7448-4ac7-8bdb-8d5400ec98f4 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:27.688895 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:27.722198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:27.736624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf60f27-e386-40c2-967c-a1ab957e2c11 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:27.786052 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fbccd7-d07c-467f-a2d7-fe5171099492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:27.787122 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbccd7-d07c-467f-a2d7-fe5171099492 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:27.787453 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbccd7-d07c-467f-a2d7-fe5171099492 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:27.787774 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbccd7-d07c-467f-a2d7-fe5171099492 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:27.788505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fbccd7-d07c-467f-a2d7-fe5171099492 None admin] RBAC: Adding query filter params (name=image) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:27.788819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fbccd7-d07c-467f-a2d7-fe5171099492 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:27.790230 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fbccd7-d07c-467f-a2d7-fe5171099492 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:27.802524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 516/1035] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:27.804089 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fbccd7-d07c-467f-a2d7-fe5171099492 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:27.822147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afc7f2-7448-4ac7-8bdb-8d5400ec98f4 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:27.866093 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf063e-6965-4147-8758-20490d35d7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:27.867219 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf063e-6965-4147-8758-20490d35d7b6 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:27.867497 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf063e-6965-4147-8758-20490d35d7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:27.867773 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf063e-6965-4147-8758-20490d35d7b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:27.875790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf063e-6965-4147-8758-20490d35d7b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:27.877359 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf063e-6965-4147-8758-20490d35d7b6 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:27.894173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-20bf063e-6965-4147-8758-20490d35d7b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:27.894173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:27.894173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:27.894173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:27.894173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:27.894173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:27.894173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:27.894173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:27.894173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:27.894173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:27.894173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:27.894173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:27.894173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:33:27.894173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:33:27.894173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:33:27.895118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:33:27.895118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:27.895118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:27.895118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:27.895118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:27.895118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:27.895118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:27.895118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:27.895118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:27.895118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:27.895118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:27.895118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:27.895118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:27.895118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:27.895118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:27.895118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:27.895118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:27.896111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:33:27.896111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:33:27.896111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:33:27.896111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:33:27.896111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:33:27.896111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:33:27.896111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:27.896111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:27.896111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 520/1036] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:33:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:27.901098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf063e-6965-4147-8758-20490d35d7b6 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:27.929269 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fbccd7-d07c-467f-a2d7-fe5171099492 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:27.970330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a49a67-42b1-4d9f-8294-6fb57c44a274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:27.971033 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a49a67-42b1-4d9f-8294-6fb57c44a274 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:27.971286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a49a67-42b1-4d9f-8294-6fb57c44a274 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:27.971598 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a49a67-42b1-4d9f-8294-6fb57c44a274 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:27.972303 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a49a67-42b1-4d9f-8294-6fb57c44a274 None admin] RBAC: Adding query filter params (type=image) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:27.972610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a49a67-42b1-4d9f-8294-6fb57c44a274 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:27.973977 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a49a67-42b1-4d9f-8294-6fb57c44a274 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:27.979679 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 517/1037] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:27.985266 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a49a67-42b1-4d9f-8294-6fb57c44a274 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:27.986854 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:28.009239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:28.028585 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf063e-6965-4147-8758-20490d35d7b6 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:28.068436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91b4ed-da6d-480f-b9fc-f6d5d6f996a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:28.069188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91b4ed-da6d-480f-b9fc-f6d5d6f996a2 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:28.069994 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91b4ed-da6d-480f-b9fc-f6d5d6f996a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:28.070375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91b4ed-da6d-480f-b9fc-f6d5d6f996a2 None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:28.071263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab91b4ed-da6d-480f-b9fc-f6d5d6f996a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:28.072021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab91b4ed-da6d-480f-b9fc-f6d5d6f996a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:28.073954 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab91b4ed-da6d-480f-b9fc-f6d5d6f996a2 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:28.094036 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 521/1038] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:33:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:28.095751 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91b4ed-da6d-480f-b9fc-f6d5d6f996a2 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:28.103401 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a49a67-42b1-4d9f-8294-6fb57c44a274 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:28.139468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb3157-7d6d-4a55-af2e-ea56769a30aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:28.140479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb3157-7d6d-4a55-af2e-ea56769a30aa None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:28.140758 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb3157-7d6d-4a55-af2e-ea56769a30aa None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:28.141026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb3157-7d6d-4a55-af2e-ea56769a30aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:28.141807 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb3157-7d6d-4a55-af2e-ea56769a30aa None admin] RBAC: Adding query filter params (interface=public, service_id=2c15332eec924b6da8d7eeab66bcc2a8, region_id=RegionOne) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:28.142763 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb3157-7d6d-4a55-af2e-ea56769a30aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:28.144101 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb3157-7d6d-4a55-af2e-ea56769a30aa None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:28.149334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 518/1039] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jun 22 11:33:27 2022] GET /identity/v3/endpoints?service_id=2c15332eec924b6da8d7eeab66bcc2a8&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:28.154953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb3157-7d6d-4a55-af2e-ea56769a30aa None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:28.205385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91b4ed-da6d-480f-b9fc-f6d5d6f996a2 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:28.243107 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:28.245191 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:28.250488 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0ac19-db7a-4a02-ae60-a3b472d06111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:28.251159 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0ac19-db7a-4a02-ae60-a3b472d06111 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:28.251999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0ac19-db7a-4a02-ae60-a3b472d06111 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:28.252268 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0ac19-db7a-4a02-ae60-a3b472d06111 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:28.253106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0ac19-db7a-4a02-ae60-a3b472d06111 None admin] RBAC: Adding query filter params (role.id=8ce71a4c1e804871b92156359277b9ee, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=479a376dcac14c509c1c55f0046d88cf) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:28.253424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0ac19-db7a-4a02-ae60-a3b472d06111 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:28.255477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0ac19-db7a-4a02-ae60-a3b472d06111 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:28.261389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 522/1040] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:33:28 2022] GET /identity/v3/role_assignments?user.id=479a376dcac14c509c1c55f0046d88cf&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=8ce71a4c1e804871b92156359277b9ee => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:28.264548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb3157-7d6d-4a55-af2e-ea56769a30aa None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:28.308376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ad9a2-56d7-447f-9fe9-922d7b817229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:28.309056 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ad9a2-56d7-447f-9fe9-922d7b817229 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:28.309339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ad9a2-56d7-447f-9fe9-922d7b817229 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:28.309612 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ad9a2-56d7-447f-9fe9-922d7b817229 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:28.310307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ad9a2-56d7-447f-9fe9-922d7b817229 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:28.311783 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ad9a2-56d7-447f-9fe9-922d7b817229 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:28.327496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 519/1041] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jun 22 11:33:28 2022] GET /identity/v3/services => generated 1957 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:33:29.802198 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322776d6-67de-4d97-98ab-b63cc71f3340 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:29.802720 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322776d6-67de-4d97-98ab-b63cc71f3340 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:29.802801 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322776d6-67de-4d97-98ab-b63cc71f3340 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:29.804295 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 523/1042] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:29.809411 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06cb24-e26a-46eb-9026-8b068cdde02f None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:29.809843 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06cb24-e26a-46eb-9026-8b068cdde02f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:29.810015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06cb24-e26a-46eb-9026-8b068cdde02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:29.827432 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-ee06cb24-e26a-46eb-9026-8b068cdde02f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:29.828618 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ee06cb24-e26a-46eb-9026-8b068cdde02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:29.899222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 520/1043] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:29.924636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322776d6-67de-4d97-98ab-b63cc71f3340 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:30.039087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322776d6-67de-4d97-98ab-b63cc71f3340 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:30.080110 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060a794d-f286-4dfb-b37a-8f543a79442a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:30.080828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a794d-f286-4dfb-b37a-8f543a79442a None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:30.081117 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a794d-f286-4dfb-b37a-8f543a79442a None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:30.081399 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060a794d-f286-4dfb-b37a-8f543a79442a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:30.082590 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a794d-f286-4dfb-b37a-8f543a79442a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:30.084067 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060a794d-f286-4dfb-b37a-8f543a79442a None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:30.085918 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 524/1044] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:30.091445 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06cb24-e26a-46eb-9026-8b068cdde02f None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:30.173382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:30.196477 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06cb24-e26a-46eb-9026-8b068cdde02f None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:30.232405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076000c-6600-4242-8035-bcf8fd442083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:30.233307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076000c-6600-4242-8035-bcf8fd442083 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:30.233773 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076000c-6600-4242-8035-bcf8fd442083 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:30.234191 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076000c-6600-4242-8035-bcf8fd442083 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:30.235041 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076000c-6600-4242-8035-bcf8fd442083 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:30.236732 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076000c-6600-4242-8035-bcf8fd442083 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:30.258533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 521/1045] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jun 22 11:33:30 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:33:30.351064 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c3e72-b55b-471c-82d7-f8b76a95c444 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:30.351387 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c3e72-b55b-471c-82d7-f8b76a95c444 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:30.351560 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c3e72-b55b-471c-82d7-f8b76a95c444 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:30.353114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 525/1046] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:30.358632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c5879-c152-4839-aa62-92e2d04a1bae None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:30.358921 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c5879-c152-4839-aa62-92e2d04a1bae None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:30.359195 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c5879-c152-4839-aa62-92e2d04a1bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:30.378952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-275c5879-c152-4839-aa62-92e2d04a1bae None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:30.380235 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-275c5879-c152-4839-aa62-92e2d04a1bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:30.411840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950afbb-a1d2-4903-a5c3-a8feb204441f None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:30.412182 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950afbb-a1d2-4903-a5c3-a8feb204441f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:30.412409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950afbb-a1d2-4903-a5c3-a8feb204441f None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:30.413806 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 526/1047] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:30.420819 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d64f0f-635d-4584-9a53-2b28219c2bc5 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:30.421179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d64f0f-635d-4584-9a53-2b28219c2bc5 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:30.421458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d64f0f-635d-4584-9a53-2b28219c2bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:30.443927 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-a6d64f0f-635d-4584-9a53-2b28219c2bc5 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:30.445141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-a6d64f0f-635d-4584-9a53-2b28219c2bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:30.461257 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 522/1048] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:30.484973 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c5879-c152-4839-aa62-92e2d04a1bae None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:30.509419 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:30.521920 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 527/1049] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:30.545532 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d64f0f-635d-4584-9a53-2b28219c2bc5 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:30.558426 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:30.600238 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c5879-c152-4839-aa62-92e2d04a1bae None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:30.640634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ad18a-65f9-44f6-b415-6bb9f4db720f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:30.641451 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ad18a-65f9-44f6-b415-6bb9f4db720f None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:30.641734 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ad18a-65f9-44f6-b415-6bb9f4db720f None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:30.642004 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ad18a-65f9-44f6-b415-6bb9f4db720f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:30.652709 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ad18a-65f9-44f6-b415-6bb9f4db720f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:33:30.658351 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d64f0f-635d-4584-9a53-2b28219c2bc5 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:30.661045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-018ad18a-65f9-44f6-b415-6bb9f4db720f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 22 11:33:30.661045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:30.661045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:30.661045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:30.661045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:30.661045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:30.661045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:30.661045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:30.661045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:30.661045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:30.661045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:30.661045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:30.661045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:33:30.661045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:33:30.661045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:33:30.661045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:33:30.661985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:33:30.661985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:33:30.661985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:33:30.661985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:33:30.661985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:30.661985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:30.661985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:30.661985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:30.661985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:30.661985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:30.661985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:30.661985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:30.661985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:30.661985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:30.661985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:30.661985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:30.662836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:30.662836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:30.662836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:30.662836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:30.662836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:33:30.662836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:33:30.662836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:33:30.662836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:33:30.662836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:33:30.662836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:33:30.662836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 22 11:33:30.662836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:30.662836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 523/1050] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jun 22 11:33:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:30.668231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ad18a-65f9-44f6-b415-6bb9f4db720f None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:30.700056 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd07ae3-5bb1-4cca-87f2-c852ecb29d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:30.700842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd07ae3-5bb1-4cca-87f2-c852ecb29d7c None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:30.701127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd07ae3-5bb1-4cca-87f2-c852ecb29d7c None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:30.701400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd07ae3-5bb1-4cca-87f2-c852ecb29d7c None admin] PATH_INFO: `/v3/services/image` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:30.702049 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd07ae3-5bb1-4cca-87f2-c852ecb29d7c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:30.703442 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd07ae3-5bb1-4cca-87f2-c852ecb29d7c None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:30.716698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-fcd07ae3-5bb1-4cca-87f2-c852ecb29d7c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 22 11:33:30.716698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:30.716698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:33:30.716698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:33:30.716698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:33:30.716698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:33:30.716698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:33:30.716698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:30.716698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 22 11:33:30.716698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:30.716698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:33:30.716698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:30.716698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:30.716698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:30.716698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:30.716698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:30.716698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:30.717742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:30.717742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:30.717742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:30.717742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:30.717742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:30.717742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:30.717742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:33:30.717742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:33:30.717742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:33:30.717742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:33:30.717742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:30.717742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:30.717742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:30.717742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:30.717742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:30.717742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:30.718592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:30.718592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:30.718592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:30.718592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:30.718592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:30.718592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:30.718592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:30.718592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:30.718592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:30.718592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:30.718592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:33:30.718592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:33:30.718592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 22 11:33:30.718592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:30.718592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 528/1051] 10.222.0.36 () {56 vars in 1105 bytes} [Wed Jun 22 11:33:30 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:30.724216 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd07ae3-5bb1-4cca-87f2-c852ecb29d7c None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:30.781071 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ad18a-65f9-44f6-b415-6bb9f4db720f None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:30.785581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:30.810630 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:30.824147 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee2767-5669-4832-aad0-9c9912823692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:30.824828 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee2767-5669-4832-aad0-9c9912823692 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:30.825118 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee2767-5669-4832-aad0-9c9912823692 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:30.825399 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee2767-5669-4832-aad0-9c9912823692 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:30.826156 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee2767-5669-4832-aad0-9c9912823692 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:30.826469 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee2767-5669-4832-aad0-9c9912823692 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:30.827882 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee2767-5669-4832-aad0-9c9912823692 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:30.832240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd07ae3-5bb1-4cca-87f2-c852ecb29d7c None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:30.836900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 524/1052] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jun 22 11:33:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:30.842981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee2767-5669-4832-aad0-9c9912823692 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:30.867702 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f740506-c7ba-4758-9bab-29ba71d6c96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:30.868555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f740506-c7ba-4758-9bab-29ba71d6c96c None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:30.868668 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f740506-c7ba-4758-9bab-29ba71d6c96c None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:30.868907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f740506-c7ba-4758-9bab-29ba71d6c96c None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:30.869604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f740506-c7ba-4758-9bab-29ba71d6c96c None admin] RBAC: Adding query filter params (name=image) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:30.869910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f740506-c7ba-4758-9bab-29ba71d6c96c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:30.871304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f740506-c7ba-4758-9bab-29ba71d6c96c None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:30.881975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 529/1053] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:30.882794 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f740506-c7ba-4758-9bab-29ba71d6c96c None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:30.960175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee2767-5669-4832-aad0-9c9912823692 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:30.997210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fc0a5-992b-44e1-ba89-3e08aa5e57cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:30.998007 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fc0a5-992b-44e1-ba89-3e08aa5e57cb None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:30.998187 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f740506-c7ba-4758-9bab-29ba71d6c96c None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:30.998298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fc0a5-992b-44e1-ba89-3e08aa5e57cb None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:30.998556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471fc0a5-992b-44e1-ba89-3e08aa5e57cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:31.011046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471fc0a5-992b-44e1-ba89-3e08aa5e57cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:31.012544 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471fc0a5-992b-44e1-ba89-3e08aa5e57cb None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:31.021000 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:31.030953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-471fc0a5-992b-44e1-ba89-3e08aa5e57cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:31.030953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:31.030953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:31.030953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:31.030953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:31.030953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:31.030953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:31.030953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:31.030953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:31.030953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:31.030953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:31.030953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:31.030953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:33:31.030953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:33:31.030953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:33:31.032388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:33:31.032388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:31.032388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:31.032388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:31.032388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:31.032388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:31.032388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:31.032388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:31.032388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:31.032388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:31.032388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:31.032388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:31.032388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:31.032388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:31.032388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:31.032388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:31.032388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:31.033409 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:33:31.033409 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:33:31.033409 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:31.033409 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:31.037106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 525/1054] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:31.037931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fc0a5-992b-44e1-ba89-3e08aa5e57cb None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:31.038572 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58f2a7-1aa9-454b-95f2-41e477e33392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:31.039253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58f2a7-1aa9-454b-95f2-41e477e33392 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:31.039574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58f2a7-1aa9-454b-95f2-41e477e33392 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:31.039861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58f2a7-1aa9-454b-95f2-41e477e33392 None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:31.040784 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58f2a7-1aa9-454b-95f2-41e477e33392 None admin] RBAC: Adding query filter params (type=image) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:31.041100 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58f2a7-1aa9-454b-95f2-41e477e33392 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:31.043419 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58f2a7-1aa9-454b-95f2-41e477e33392 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:31.054270 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 530/1055] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:31.055127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58f2a7-1aa9-454b-95f2-41e477e33392 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:31.082045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:31.149871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471fc0a5-992b-44e1-ba89-3e08aa5e57cb None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:31.162826 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58f2a7-1aa9-454b-95f2-41e477e33392 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:31.187844 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54025433-dcd2-49c6-821d-e36a020e7527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:31.188509 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54025433-dcd2-49c6-821d-e36a020e7527 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:31.188789 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54025433-dcd2-49c6-821d-e36a020e7527 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:31.189101 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54025433-dcd2-49c6-821d-e36a020e7527 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:31.189826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54025433-dcd2-49c6-821d-e36a020e7527 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:31.190137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54025433-dcd2-49c6-821d-e36a020e7527 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:31.191524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54025433-dcd2-49c6-821d-e36a020e7527 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:31.200005 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93132e90-b91d-4547-b737-9dd148f6ad0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:31.200596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93132e90-b91d-4547-b737-9dd148f6ad0f None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:31.200877 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93132e90-b91d-4547-b737-9dd148f6ad0f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:31.201151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93132e90-b91d-4547-b737-9dd148f6ad0f None admin] PATH_INFO: `/v3/endpoints` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:31.201885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93132e90-b91d-4547-b737-9dd148f6ad0f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:31.203386 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 526/1056] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:31.203515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93132e90-b91d-4547-b737-9dd148f6ad0f None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:31.208917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54025433-dcd2-49c6-821d-e36a020e7527 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:31.231677 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 531/1057] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jun 22 11:33:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:33:31.324460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:31.330214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54025433-dcd2-49c6-821d-e36a020e7527 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:31.374942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbba26-95a1-42c4-876e-409402461980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:31.375769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbba26-95a1-42c4-876e-409402461980 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:31.376061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbba26-95a1-42c4-876e-409402461980 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:31.376335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbba26-95a1-42c4-876e-409402461980 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:31.385705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbba26-95a1-42c4-876e-409402461980 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:31.387103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbba26-95a1-42c4-876e-409402461980 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:31.401600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-ebcbba26-95a1-42c4-876e-409402461980 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 22 11:33:31.401600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:31.401600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:31.401600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:31.401600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:31.401600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:31.401600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:31.401600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:31.401600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:31.401600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:31.401600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:31.401600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:31.401600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:33:31.401600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:33:31.401600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:33:31.401600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:33:31.402781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:31.402781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:31.402781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:33:31.402781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:31.402781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:33:31.402781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:31.402781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:31.402781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:31.402781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:31.402781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:31.402781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:31.402781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:31.402781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:31.402781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:31.402781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:31.402781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:31.402781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:31.402781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:31.403851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:31.403851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:31.403851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:33:31.403851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:33:31.403851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:33:31.403851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:33:31.403851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:33:31.403851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:33:31.403851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 22 11:33:31.403851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:31.403851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 527/1058] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jun 22 11:33:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:31.408805 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93132e90-b91d-4547-b737-9dd148f6ad0f None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:31.458887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:31.522322 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93132e90-b91d-4547-b737-9dd148f6ad0f None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:31.558946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843facc8-ebac-49bc-87fa-08b973ea1d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:31.559648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843facc8-ebac-49bc-87fa-08b973ea1d61 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:31.559926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843facc8-ebac-49bc-87fa-08b973ea1d61 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:31.560200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843facc8-ebac-49bc-87fa-08b973ea1d61 None admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:31.561030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843facc8-ebac-49bc-87fa-08b973ea1d61 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:31.561365 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843facc8-ebac-49bc-87fa-08b973ea1d61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:31.562762 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843facc8-ebac-49bc-87fa-08b973ea1d61 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:31.580723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 532/1059] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jun 22 11:33:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:31.586046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbba26-95a1-42c4-876e-409402461980 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:31.687062 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbba26-95a1-42c4-876e-409402461980 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:31.722649 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf0a53-a1a0-4a71-953d-444ef1c5d3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:31.722649 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf0a53-a1a0-4a71-953d-444ef1c5d3e2 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:31.722649 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf0a53-a1a0-4a71-953d-444ef1c5d3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:31.722649 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf0a53-a1a0-4a71-953d-444ef1c5d3e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:31.733310 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf0a53-a1a0-4a71-953d-444ef1c5d3e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:31.734723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf0a53-a1a0-4a71-953d-444ef1c5d3e2 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:31.751899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-6ddf0a53-a1a0-4a71-953d-444ef1c5d3e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:31.751899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:31.751899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:31.751899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:31.751899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:31.751899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:31.751899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:31.751899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:31.751899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:31.751899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:31.751899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:31.751899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:31.751899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:33:31.751899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:33:31.751899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:33:31.752915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:33:31.752915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:31.752915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:31.752915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:31.752915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:31.752915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:31.752915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:31.752915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:31.752915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:31.752915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:31.752915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:31.752915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:31.752915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:31.752915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:31.752915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:31.752915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:31.752915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:31.753911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:33:31.753911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:33:31.753911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:31.753911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:31.755719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 528/1060] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:31.758594 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843facc8-ebac-49bc-87fa-08b973ea1d61 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:31.855739 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:31.863001 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843facc8-ebac-49bc-87fa-08b973ea1d61 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:31.898034 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e00f6b-fcb1-49af-9ab0-35bb2732825b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:31.898685 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e00f6b-fcb1-49af-9ab0-35bb2732825b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:31.898965 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e00f6b-fcb1-49af-9ab0-35bb2732825b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:31.899237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e00f6b-fcb1-49af-9ab0-35bb2732825b None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:31.899996 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e00f6b-fcb1-49af-9ab0-35bb2732825b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:31.900301 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e00f6b-fcb1-49af-9ab0-35bb2732825b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:31.901687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e00f6b-fcb1-49af-9ab0-35bb2732825b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:31.914947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 533/1061] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:31.920259 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf0a53-a1a0-4a71-953d-444ef1c5d3e2 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:31.932249 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:32.030468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf0a53-a1a0-4a71-953d-444ef1c5d3e2 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:32.064570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ea47a-cff0-4cbf-81ed-e1656dc140e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:32.065361 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ea47a-cff0-4cbf-81ed-e1656dc140e3 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:32.065644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ea47a-cff0-4cbf-81ed-e1656dc140e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:32.065918 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ea47a-cff0-4cbf-81ed-e1656dc140e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:32.072672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ea47a-cff0-4cbf-81ed-e1656dc140e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:32.074124 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ea47a-cff0-4cbf-81ed-e1656dc140e3 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:32.088356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-3d0ea47a-cff0-4cbf-81ed-e1656dc140e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:32.088356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:32.088356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:32.088356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:32.088356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:32.088356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:32.088356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:32.088356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:32.088356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:32.088356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:32.088356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:32.088356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:32.088356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:33:32.088356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:33:32.088356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:33:32.089287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:33:32.089287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:32.089287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:32.089287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:32.089287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:32.089287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:32.089287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:32.089287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:32.089287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:32.089287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:32.089287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:32.089287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:32.089287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:32.089287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:32.089287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:32.089287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:32.089287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:32.095001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:33:32.095001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:33:32.095001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:33:32.095001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:33:32.095001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:33:32.095001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:33:32.095001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:32.095001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:32.095001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 529/1062] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:33:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:32.096241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e00f6b-fcb1-49af-9ab0-35bb2732825b None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:32.221152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e00f6b-fcb1-49af-9ab0-35bb2732825b None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:32.265520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4980f-bf3a-4c62-ab8c-4de52448be9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:32.266197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4980f-bf3a-4c62-ab8c-4de52448be9a None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:32.266484 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4980f-bf3a-4c62-ab8c-4de52448be9a None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:32.266762 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4980f-bf3a-4c62-ab8c-4de52448be9a None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:32.268139 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4980f-bf3a-4c62-ab8c-4de52448be9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:32.268445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4980f-bf3a-4c62-ab8c-4de52448be9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:32.269898 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4980f-bf3a-4c62-ab8c-4de52448be9a None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:32.287802 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 534/1063] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:33:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:32.289003 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ea47a-cff0-4cbf-81ed-e1656dc140e3 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:32.357988 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:32.397269 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ea47a-cff0-4cbf-81ed-e1656dc140e3 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:32.409966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81e0ba-c634-43a9-80b0-2f19938f0e96 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:32.410423 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81e0ba-c634-43a9-80b0-2f19938f0e96 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:32.410717 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81e0ba-c634-43a9-80b0-2f19938f0e96 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:32.412778 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 535/1064] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:32.417961 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b76802-d34b-411d-a76f-82cbc0b2d99d None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:32.418251 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b76802-d34b-411d-a76f-82cbc0b2d99d None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:32.418532 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b76802-d34b-411d-a76f-82cbc0b2d99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:32.435115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504a473-7e39-4067-9da5-52f52b9d642e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:32.436036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504a473-7e39-4067-9da5-52f52b9d642e None admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:32.436319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504a473-7e39-4067-9da5-52f52b9d642e None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:32.436597 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504a473-7e39-4067-9da5-52f52b9d642e None admin] PATH_INFO: `/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/479a376dcac14c509c1c55f0046d88cf/roles/8ce71a4c1e804871b92156359277b9ee` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:32.438825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504a473-7e39-4067-9da5-52f52b9d642e None admin] RBAC: Authorizing `identity:create_grant(project_id=47d2bb094c9843c9a7726f1d672d2433, user_id=479a376dcac14c509c1c55f0046d88cf, role_id=8ce71a4c1e804871b92156359277b9ee)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:32.440032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-92b76802-d34b-411d-a76f-82cbc0b2d99d None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:32.440874 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504a473-7e39-4067-9da5-52f52b9d642e None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:32.441233 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-92b76802-d34b-411d-a76f-82cbc0b2d99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:32.467468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 530/1065] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:33:32 2022] PUT /identity/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/479a376dcac14c509c1c55f0046d88cf/roles/8ce71a4c1e804871b92156359277b9ee => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:33:32.467722 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:32.540172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 536/1066] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:32.564863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504a473-7e39-4067-9da5-52f52b9d642e None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:32.676142 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504a473-7e39-4067-9da5-52f52b9d642e None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:32.700991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:32.711846 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2e7be-4373-4ad8-baf1-fb7d59796f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:32.712637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2e7be-4373-4ad8-baf1-fb7d59796f8b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:32.712917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2e7be-4373-4ad8-baf1-fb7d59796f8b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:32.713191 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2e7be-4373-4ad8-baf1-fb7d59796f8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:32.724574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2e7be-4373-4ad8-baf1-fb7d59796f8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:33:32.732864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-88c2e7be-4373-4ad8-baf1-fb7d59796f8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:33:32.732864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:32.732864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:32.732864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:32.732864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:32.732864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:32.732864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:32.732864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:32.732864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:32.732864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:32.732864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:32.732864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:32.732864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:33:32.732864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:33:32.732864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:33:32.732864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:33:32.733919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:33:32.733919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:33:32.733919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:33:32.733919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:33:32.733919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:32.733919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:32.733919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:32.733919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:32.733919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:32.733919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:32.733919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:32.733919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:32.733919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:32.733919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:32.733919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:32.733919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:32.734916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:32.734916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:32.734916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:32.734916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:32.734916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:33:32.734916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:33:32.734916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:33:32.734916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:33:32.734916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:33:32.734916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:33:32.734916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:33:32.734916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:32.736779 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 531/1067] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:33:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:32.741301 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b76802-d34b-411d-a76f-82cbc0b2d99d None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:32.841823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b76802-d34b-411d-a76f-82cbc0b2d99d None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:32.876486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d2cade-a89d-4c9a-95a6-347b474e943e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:32.877143 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2cade-a89d-4c9a-95a6-347b474e943e None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:32.877425 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2cade-a89d-4c9a-95a6-347b474e943e None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:32.877735 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2cade-a89d-4c9a-95a6-347b474e943e None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:32.878589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d2cade-a89d-4c9a-95a6-347b474e943e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:32.878895 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d2cade-a89d-4c9a-95a6-347b474e943e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:32.880300 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d2cade-a89d-4c9a-95a6-347b474e943e None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:32.894261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 537/1068] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:33:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:32.895573 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2e7be-4373-4ad8-baf1-fb7d59796f8b None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:33.008140 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2e7be-4373-4ad8-baf1-fb7d59796f8b None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:33.041718 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b5601-c9c0-4fc4-928f-c35b27caeff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:33.042586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b5601-c9c0-4fc4-928f-c35b27caeff9 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:33.042866 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b5601-c9c0-4fc4-928f-c35b27caeff9 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:33.043144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b5601-c9c0-4fc4-928f-c35b27caeff9 None admin] PATH_INFO: `/v3/users/cfd2600e42f54f37aad17d0fdddd51d8` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:33.057248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b5601-c9c0-4fc4-928f-c35b27caeff9 None admin] RBAC: Authorizing `identity:get_user(user_id=cfd2600e42f54f37aad17d0fdddd51d8)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:33.058691 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b5601-c9c0-4fc4-928f-c35b27caeff9 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:33.060627 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 532/1069] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:33:32 2022] GET /identity/v3/users/cfd2600e42f54f37aad17d0fdddd51d8 => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:33.066716 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d2cade-a89d-4c9a-95a6-347b474e943e None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:33.079555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:33.171651 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d2cade-a89d-4c9a-95a6-347b474e943e None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:33.209640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dcae4f-6cfb-4de0-8b97-62361640086c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:33.210291 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dcae4f-6cfb-4de0-8b97-62361640086c None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:33.210601 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dcae4f-6cfb-4de0-8b97-62361640086c None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:33.210891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dcae4f-6cfb-4de0-8b97-62361640086c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:33.281106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dcae4f-6cfb-4de0-8b97-62361640086c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:33.283101 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dcae4f-6cfb-4de0-8b97-62361640086c None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:33.294508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:33.302426 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d7095-9444-42fb-a23f-3d1edbd893d0 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:33.302844 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d7095-9444-42fb-a23f-3d1edbd893d0 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:33.303127 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d7095-9444-42fb-a23f-3d1edbd893d0 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:33.304790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 533/1070] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:33.310354 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7ab19-2198-49d2-9163-e56b7e5d0f74 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:33.310991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7ab19-2198-49d2-9163-e56b7e5d0f74 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:33.311277 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d7ab19-2198-49d2-9163-e56b7e5d0f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:33.329880 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-17d7ab19-2198-49d2-9163-e56b7e5d0f74 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:33.332490 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-17d7ab19-2198-49d2-9163-e56b7e5d0f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:33.382717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:33.395028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 538/1071] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:33:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 22 11:33:33.400867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dcae4f-6cfb-4de0-8b97-62361640086c None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:33.420358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 534/1072] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:33:33.443234 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d7ab19-2198-49d2-9163-e56b7e5d0f74 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:33.512797 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dcae4f-6cfb-4de0-8b97-62361640086c None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:33.526255 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:33.558228 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ecf72a-aa47-4b2a-afda-c81a111b2ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:33.559529 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ecf72a-aa47-4b2a-afda-c81a111b2ddc None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:33.560051 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ecf72a-aa47-4b2a-afda-c81a111b2ddc None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:33.560544 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ecf72a-aa47-4b2a-afda-c81a111b2ddc None admin] PATH_INFO: `/v3/projects/2504616647f74ee6b91a718f458eb9a2` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:33.562680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ecf72a-aa47-4b2a-afda-c81a111b2ddc None admin] RBAC: Authorizing `identity:get_project(project_id=2504616647f74ee6b91a718f458eb9a2)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:33.565196 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d7ab19-2198-49d2-9163-e56b7e5d0f74 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:33.565558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ecf72a-aa47-4b2a-afda-c81a111b2ddc None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:33.573550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 539/1073] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:33:33 2022] GET /identity/v3/projects/2504616647f74ee6b91a718f458eb9a2 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:33.576969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ecf72a-aa47-4b2a-afda-c81a111b2ddc None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:33.609230 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc85b4-3298-4d0c-8574-e41dfac6c8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:33.610249 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc85b4-3298-4d0c-8574-e41dfac6c8d7 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:33.610554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc85b4-3298-4d0c-8574-e41dfac6c8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:33.611639 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc85b4-3298-4d0c-8574-e41dfac6c8d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:33.656985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:33.685820 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc85b4-3298-4d0c-8574-e41dfac6c8d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:33.687611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effc85b4-3298-4d0c-8574-e41dfac6c8d7 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:33.689796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ecf72a-aa47-4b2a-afda-c81a111b2ddc None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:33.725894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd5c46-9f5e-4815-953a-10ec202868de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:33.726540 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd5c46-9f5e-4815-953a-10ec202868de None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:33.726830 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd5c46-9f5e-4815-953a-10ec202868de None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:33.727110 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd5c46-9f5e-4815-953a-10ec202868de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:33.728028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd5c46-9f5e-4815-953a-10ec202868de None admin] RBAC: Adding query filter params (role.id=96c584428d924fdba9671949f760c406, scope.project.id=2504616647f74ee6b91a718f458eb9a2, user.id=cfd2600e42f54f37aad17d0fdddd51d8) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:33.728350 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd5c46-9f5e-4815-953a-10ec202868de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:33.729787 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd5c46-9f5e-4815-953a-10ec202868de None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:33.735960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 540/1074] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:33:33 2022] GET /identity/v3/role_assignments?user.id=cfd2600e42f54f37aad17d0fdddd51d8&scope.project.id=2504616647f74ee6b91a718f458eb9a2&role.id=96c584428d924fdba9671949f760c406 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:33.802726 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 535/1075] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:33:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 22 11:33:33.809431 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd5c46-9f5e-4815-953a-10ec202868de None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:33.866673 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:33.931447 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd5c46-9f5e-4815-953a-10ec202868de None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:33.971201 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18b59a-ca2a-486f-8c94-58017d6a76cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:33.972077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18b59a-ca2a-486f-8c94-58017d6a76cf None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:33.972365 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18b59a-ca2a-486f-8c94-58017d6a76cf None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:33.972646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18b59a-ca2a-486f-8c94-58017d6a76cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:33.983303 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18b59a-ca2a-486f-8c94-58017d6a76cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:33.984759 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18b59a-ca2a-486f-8c94-58017d6a76cf None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:34.003098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-df18b59a-ca2a-486f-8c94-58017d6a76cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:34.003098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:34.003098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:34.003098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:34.003098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:34.003098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:34.003098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:34.003098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:34.003098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:34.003098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:34.003098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:34.003098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:34.003098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:33:34.003098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:33:34.003098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:33:34.004259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:33:34.004259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:34.004259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:34.004259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:34.004259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:34.004259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:34.004259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:34.004259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:34.004259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:34.004259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:34.004259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:34.004259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:34.004259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:34.004259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:34.004259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:34.004259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:34.004259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:34.009913 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:33:34.009913 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:33:34.009913 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:34.009913 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:34.009913 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 541/1076] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:34.011974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc85b4-3298-4d0c-8574-e41dfac6c8d7 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:34.090785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:34.116151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc85b4-3298-4d0c-8574-e41dfac6c8d7 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:34.151855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950a073-e079-4e0a-9e9f-207beda137f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:34.152518 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950a073-e079-4e0a-9e9f-207beda137f0 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:34.152798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950a073-e079-4e0a-9e9f-207beda137f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:34.153072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950a073-e079-4e0a-9e9f-207beda137f0 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:34.153784 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950a073-e079-4e0a-9e9f-207beda137f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:34.154088 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950a073-e079-4e0a-9e9f-207beda137f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:34.155476 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950a073-e079-4e0a-9e9f-207beda137f0 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:34.171431 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 536/1077] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:34.537277 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101e1ff-f3fe-4861-848d-6353e986826c None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:34.537559 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101e1ff-f3fe-4861-848d-6353e986826c None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:34.537837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101e1ff-f3fe-4861-848d-6353e986826c None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:34.539299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 542/1078] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:34.545126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc1f5a-9cc0-4e3f-884d-4313815711a5 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:34.545495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc1f5a-9cc0-4e3f-884d-4313815711a5 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:34.545778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc1f5a-9cc0-4e3f-884d-4313815711a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:34.562164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-8edc1f5a-9cc0-4e3f-884d-4313815711a5 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:34.563370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-8edc1f5a-9cc0-4e3f-884d-4313815711a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:34.629491 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 537/1079] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:34.653011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101e1ff-f3fe-4861-848d-6353e986826c None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:34.745794 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:34.754319 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101e1ff-f3fe-4861-848d-6353e986826c None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:34.789874 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e40f2-3bc7-4a74-ab2c-7e61ec4c12fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:34.790596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e40f2-3bc7-4a74-ab2c-7e61ec4c12fe None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:34.790877 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e40f2-3bc7-4a74-ab2c-7e61ec4c12fe None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:34.791152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2e40f2-3bc7-4a74-ab2c-7e61ec4c12fe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:34.802054 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2e40f2-3bc7-4a74-ab2c-7e61ec4c12fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:33:34.810616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-3d2e40f2-3bc7-4a74-ab2c-7e61ec4c12fe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 22 11:33:34.810616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:34.810616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:34.810616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:34.810616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:34.810616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:34.810616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:34.810616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:34.810616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:34.810616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:34.810616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:34.810616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:34.810616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:33:34.810616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:33:34.810616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:33:34.810616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:33:34.811745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:33:34.811745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:33:34.811745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:33:34.811745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:33:34.811745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:34.811745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:34.811745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:34.811745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:34.811745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:34.811745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:34.811745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:34.811745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:34.811745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:34.811745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:34.811745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:34.811745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:34.812793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:34.812793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:34.812793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:34.812793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:34.812793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:33:34.812793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:33:34.812793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:33:34.812793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:33:34.812793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:33:34.812793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:33:34.812793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 22 11:33:34.812793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:34.812793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 543/1080] 10.222.0.36 () {56 vars in 1120 bytes} [Wed Jun 22 11:33:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:34.818219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc1f5a-9cc0-4e3f-884d-4313815711a5 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:34.881865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:34.919544 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc1f5a-9cc0-4e3f-884d-4313815711a5 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:34.953510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56eb88-f0a8-4c28-aebf-53f3cb2a1613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:34.954185 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56eb88-f0a8-4c28-aebf-53f3cb2a1613 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:34.954501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56eb88-f0a8-4c28-aebf-53f3cb2a1613 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:34.954806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56eb88-f0a8-4c28-aebf-53f3cb2a1613 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:34.955638 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56eb88-f0a8-4c28-aebf-53f3cb2a1613 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:34.955982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56eb88-f0a8-4c28-aebf-53f3cb2a1613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:34.957358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56eb88-f0a8-4c28-aebf-53f3cb2a1613 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:34.965572 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 538/1081] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jun 22 11:33:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:34.971740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e40f2-3bc7-4a74-ab2c-7e61ec4c12fe None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:35.068939 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2e40f2-3bc7-4a74-ab2c-7e61ec4c12fe None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:35.101182 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b74bf1-589b-40ef-9589-df1bf1b63bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:35.101963 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b74bf1-589b-40ef-9589-df1bf1b63bc9 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:35.102247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b74bf1-589b-40ef-9589-df1bf1b63bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:35.102522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b74bf1-589b-40ef-9589-df1bf1b63bc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:35.112882 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b74bf1-589b-40ef-9589-df1bf1b63bc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:35.114325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b74bf1-589b-40ef-9589-df1bf1b63bc9 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:35.135951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-61b74bf1-589b-40ef-9589-df1bf1b63bc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:35.135951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:35.135951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:35.135951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:35.135951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:35.135951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:35.135951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:35.135951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:35.135951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:35.135951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:35.135951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:35.135951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:35.135951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:33:35.135951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:33:35.135951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:33:35.137047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:33:35.137047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:35.137047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:35.137047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:35.137047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:35.137047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:35.137047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:35.137047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:35.137047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:35.137047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:35.137047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:35.137047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:35.137047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:35.137047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:35.137047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:35.137047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:35.137047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:35.140529 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:33:35.140529 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:33:35.140529 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:35.140529 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:35.140529 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 544/1082] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:35.144804 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56eb88-f0a8-4c28-aebf-53f3cb2a1613 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:35.243450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56eb88-f0a8-4c28-aebf-53f3cb2a1613 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:35.268553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:35.277918 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35b85f-40b0-4fcd-a1de-00bc66f46539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:35.278584 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35b85f-40b0-4fcd-a1de-00bc66f46539 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:35.278871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35b85f-40b0-4fcd-a1de-00bc66f46539 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:35.279148 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35b85f-40b0-4fcd-a1de-00bc66f46539 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:35.279910 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35b85f-40b0-4fcd-a1de-00bc66f46539 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:35.280219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35b85f-40b0-4fcd-a1de-00bc66f46539 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:35.281588 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35b85f-40b0-4fcd-a1de-00bc66f46539 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:35.293968 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 539/1083] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:35.299618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b74bf1-589b-40ef-9589-df1bf1b63bc9 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:35.308901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:35.399916 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b74bf1-589b-40ef-9589-df1bf1b63bc9 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:35.435536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35317412-4e6b-4e91-86e5-03648877f0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:35.436330 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35317412-4e6b-4e91-86e5-03648877f0e2 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:35.436615 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35317412-4e6b-4e91-86e5-03648877f0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:35.436892 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35317412-4e6b-4e91-86e5-03648877f0e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:35.445028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35317412-4e6b-4e91-86e5-03648877f0e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:35.446485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35317412-4e6b-4e91-86e5-03648877f0e2 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:35.460833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-35317412-4e6b-4e91-86e5-03648877f0e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 22 11:33:35.460833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:35.460833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:35.460833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:35.460833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:35.460833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:35.460833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:35.460833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:35.460833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:35.460833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:35.460833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:35.460833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:35.460833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:33:35.460833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:33:35.460833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:33:35.460833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:33:35.462019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:35.462019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:35.462019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:33:35.462019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:35.462019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:33:35.462019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:33:35.462019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:35.462019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:35.462019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:35.462019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:35.462019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:35.462019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:35.462019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:35.462019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:35.462019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:35.462019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:35.462019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:35.462019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:35.463073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:35.463073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:35.463073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:33:35.463073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:33:35.463073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:33:35.463073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:33:35.463073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:33:35.463073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:33:35.463073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 22 11:33:35.463073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:35.463073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 545/1084] 10.222.0.36 () {56 vars in 1093 bytes} [Wed Jun 22 11:33:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:35.475538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35b85f-40b0-4fcd-a1de-00bc66f46539 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:35.586408 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35b85f-40b0-4fcd-a1de-00bc66f46539 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:35.619586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f94f0-afb3-4be4-ada4-f329ec42650a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:35.620255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f94f0-afb3-4be4-ada4-f329ec42650a None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:35.620539 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f94f0-afb3-4be4-ada4-f329ec42650a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:35.620819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f94f0-afb3-4be4-ada4-f329ec42650a None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:35.621665 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f94f0-afb3-4be4-ada4-f329ec42650a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:35.621975 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f94f0-afb3-4be4-ada4-f329ec42650a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:35.623408 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f94f0-afb3-4be4-ada4-f329ec42650a None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:35.640980 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 540/1085] 10.222.0.36 () {56 vars in 1133 bytes} [Wed Jun 22 11:33:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:35.647273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35317412-4e6b-4e91-86e5-03648877f0e2 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:35.712046 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:35.748350 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35317412-4e6b-4e91-86e5-03648877f0e2 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:35.754289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eabb90-9f5e-4dbd-b9d1-fa59ff641a49 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:35.754577 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eabb90-9f5e-4dbd-b9d1-fa59ff641a49 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:35.754852 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eabb90-9f5e-4dbd-b9d1-fa59ff641a49 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:35.756282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 541/1086] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:33:35.761511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12defa03-9fc7-4232-82ab-a8eabcf6e596 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:35.761796 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12defa03-9fc7-4232-82ab-a8eabcf6e596 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:35.762072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12defa03-9fc7-4232-82ab-a8eabcf6e596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:35.779305 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-12defa03-9fc7-4232-82ab-a8eabcf6e596 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:35.780521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-12defa03-9fc7-4232-82ab-a8eabcf6e596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:35.788014 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c53c45-8f9a-4c5d-b42b-092d11a7db58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:35.788593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c53c45-8f9a-4c5d-b42b-092d11a7db58 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:35.788879 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c53c45-8f9a-4c5d-b42b-092d11a7db58 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:35.789163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c53c45-8f9a-4c5d-b42b-092d11a7db58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:35.821286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:35.850615 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 542/1087] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:33:35.858166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c53c45-8f9a-4c5d-b42b-092d11a7db58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:35.859653 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c53c45-8f9a-4c5d-b42b-092d11a7db58 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:35.874076 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12defa03-9fc7-4232-82ab-a8eabcf6e596 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:35.924022 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:35.959881 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 546/1088] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:33:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 22 11:33:35.965686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c53c45-8f9a-4c5d-b42b-092d11a7db58 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:35.975872 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12defa03-9fc7-4232-82ab-a8eabcf6e596 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:36.010540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3232c9bb-3729-499d-afb1-e32271ed36d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:36.011322 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232c9bb-3729-499d-afb1-e32271ed36d7 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:36.011664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232c9bb-3729-499d-afb1-e32271ed36d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:36.011939 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232c9bb-3729-499d-afb1-e32271ed36d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:36.022167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3232c9bb-3729-499d-afb1-e32271ed36d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:33:36.029663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-3232c9bb-3729-499d-afb1-e32271ed36d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:33:36.029663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:36.029663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:36.029663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:36.029663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:36.029663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:36.029663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:36.029663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:36.029663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:36.029663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:36.029663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:36.029663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:36.029663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:33:36.029663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:33:36.029663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:33:36.029663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:33:36.030800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:33:36.030800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:33:36.030800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:33:36.030800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:33:36.030800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:36.030800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:36.030800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:36.030800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:36.030800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:36.030800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:36.030800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:36.030800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:36.030800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:36.030800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:36.030800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:36.030800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:36.031855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:36.031855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:36.031855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:36.031855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:36.031855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:33:36.031855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:33:36.031855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:33:36.031855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:33:36.031855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:33:36.031855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:33:36.031855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:33:36.031855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:36.031855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 543/1089] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:33:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:36.037032 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3232c9bb-3729-499d-afb1-e32271ed36d7 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:36.066910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c53c45-8f9a-4c5d-b42b-092d11a7db58 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:36.072033 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:36.100999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148440d6-f165-4abb-827f-4770eac4811c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:36.101781 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148440d6-f165-4abb-827f-4770eac4811c None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:36.102067 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148440d6-f165-4abb-827f-4770eac4811c None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:36.102341 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148440d6-f165-4abb-827f-4770eac4811c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:36.112912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148440d6-f165-4abb-827f-4770eac4811c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:36.114403 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148440d6-f165-4abb-827f-4770eac4811c None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:36.132202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-148440d6-f165-4abb-827f-4770eac4811c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:36.132202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:36.132202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:36.132202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:36.132202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:36.132202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:36.132202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:36.132202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:36.132202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:36.132202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:36.132202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:36.132202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:36.132202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:33:36.132202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:33:36.132202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:33:36.133397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:33:36.133397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:36.133397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:36.133397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:36.133397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:36.133397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:36.133397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:36.133397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:36.133397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:36.133397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:36.133397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:36.133397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:36.133397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:36.133397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:36.133397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:36.133397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:36.133397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:36.134487 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:33:36.134487 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:33:36.134487 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:33:36.134487 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:36.134487 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 547/1090] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:36.139061 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148440d6-f165-4abb-827f-4770eac4811c None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:36.141665 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3232c9bb-3729-499d-afb1-e32271ed36d7 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:36.157709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:36.175790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb06df2-fda3-4ce7-889c-2cb3cf66e3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:36.176457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb06df2-fda3-4ce7-889c-2cb3cf66e3c4 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:36.176740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb06df2-fda3-4ce7-889c-2cb3cf66e3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:36.177019 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb06df2-fda3-4ce7-889c-2cb3cf66e3c4 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:36.177766 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb06df2-fda3-4ce7-889c-2cb3cf66e3c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:36.178069 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb06df2-fda3-4ce7-889c-2cb3cf66e3c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:36.179454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb06df2-fda3-4ce7-889c-2cb3cf66e3c4 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:36.187523 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 544/1091] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:33:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:36.193766 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb06df2-fda3-4ce7-889c-2cb3cf66e3c4 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:36.238970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148440d6-f165-4abb-827f-4770eac4811c None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:36.273707 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3beea44-3dec-45eb-ac6d-cd68756dee3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:36.274370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3beea44-3dec-45eb-ac6d-cd68756dee3a None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:36.274657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3beea44-3dec-45eb-ac6d-cd68756dee3a None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:36.274933 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3beea44-3dec-45eb-ac6d-cd68756dee3a None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:36.275823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3beea44-3dec-45eb-ac6d-cd68756dee3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:36.276138 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3beea44-3dec-45eb-ac6d-cd68756dee3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:36.277514 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3beea44-3dec-45eb-ac6d-cd68756dee3a None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:36.288419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:36.289053 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 548/1092] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:36.294922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3beea44-3dec-45eb-ac6d-cd68756dee3a None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:36.295150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb06df2-fda3-4ce7-889c-2cb3cf66e3c4 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:36.330151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e055f93-955c-40d8-8ba8-9110bc29cec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:36.331009 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e055f93-955c-40d8-8ba8-9110bc29cec2 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:36.331286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e055f93-955c-40d8-8ba8-9110bc29cec2 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:36.331579 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e055f93-955c-40d8-8ba8-9110bc29cec2 None admin] PATH_INFO: `/v3/users/cfd2600e42f54f37aad17d0fdddd51d8` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:36.332763 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e055f93-955c-40d8-8ba8-9110bc29cec2 None admin] RBAC: Authorizing `identity:get_user(user_id=cfd2600e42f54f37aad17d0fdddd51d8)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:36.334175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e055f93-955c-40d8-8ba8-9110bc29cec2 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:36.335855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 545/1093] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:33:36 2022] GET /identity/v3/users/cfd2600e42f54f37aad17d0fdddd51d8 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:36.341478 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e055f93-955c-40d8-8ba8-9110bc29cec2 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:36.374667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:36.397653 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3beea44-3dec-45eb-ac6d-cd68756dee3a None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:36.431782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a167e9-60df-4e3f-b76b-574037d3780f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:36.432582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a167e9-60df-4e3f-b76b-574037d3780f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:36.432866 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a167e9-60df-4e3f-b76b-574037d3780f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:36.433141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a167e9-60df-4e3f-b76b-574037d3780f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:36.439717 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a167e9-60df-4e3f-b76b-574037d3780f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:36.441161 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a167e9-60df-4e3f-b76b-574037d3780f None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:36.442785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e055f93-955c-40d8-8ba8-9110bc29cec2 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:36.455887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-d0a167e9-60df-4e3f-b76b-574037d3780f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:36.455887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:36.455887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:36.455887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:36.455887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:36.455887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:36.455887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:36.455887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:36.455887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:36.455887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:36.455887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:36.455887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:36.455887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:33:36.455887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:33:36.455887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:33:36.456861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:33:36.456861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:36.456861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:36.456861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:36.456861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:36.456861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:36.456861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:36.456861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:36.456861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:36.456861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:36.456861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:36.456861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:36.456861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:36.456861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:36.456861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:36.456861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:36.456861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:36.458207 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:33:36.458207 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:33:36.458207 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:33:36.458207 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:33:36.458207 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:33:36.458207 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:33:36.458207 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:33:36.458207 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:33:36.458207 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 549/1094] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:33:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:36.462769 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a167e9-60df-4e3f-b76b-574037d3780f None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:36.478777 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355837a3-7a41-40a2-bb8a-08c6708c1399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:36.479672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355837a3-7a41-40a2-bb8a-08c6708c1399 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:36.479959 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355837a3-7a41-40a2-bb8a-08c6708c1399 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:36.480236 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355837a3-7a41-40a2-bb8a-08c6708c1399 None admin] PATH_INFO: `/v3/projects/2504616647f74ee6b91a718f458eb9a2` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:36.481410 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355837a3-7a41-40a2-bb8a-08c6708c1399 None admin] RBAC: Authorizing `identity:get_project(project_id=2504616647f74ee6b91a718f458eb9a2)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:36.482825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355837a3-7a41-40a2-bb8a-08c6708c1399 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:36.484561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 546/1095] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:33:36 2022] GET /identity/v3/projects/2504616647f74ee6b91a718f458eb9a2 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:36.490965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355837a3-7a41-40a2-bb8a-08c6708c1399 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:36.521450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:36.568497 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a167e9-60df-4e3f-b76b-574037d3780f None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:36.593174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355837a3-7a41-40a2-bb8a-08c6708c1399 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:36.604055 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2502f-2650-4adb-88ac-1949c19a90cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:36.604813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2502f-2650-4adb-88ac-1949c19a90cd None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:36.605198 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2502f-2650-4adb-88ac-1949c19a90cd None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:36.605459 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2502f-2650-4adb-88ac-1949c19a90cd None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:36.606240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b2502f-2650-4adb-88ac-1949c19a90cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:36.606547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b2502f-2650-4adb-88ac-1949c19a90cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:36.608076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b2502f-2650-4adb-88ac-1949c19a90cd None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:36.615300 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:36.622294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 550/1096] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:33:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:36.627887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2502f-2650-4adb-88ac-1949c19a90cd None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:36.628967 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec5248-cad7-4e92-ab52-8120e6113fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:36.629853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec5248-cad7-4e92-ab52-8120e6113fb4 None admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:36.630132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec5248-cad7-4e92-ab52-8120e6113fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:36.630452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec5248-cad7-4e92-ab52-8120e6113fb4 None admin] PATH_INFO: `/v3/projects/2504616647f74ee6b91a718f458eb9a2/users/cfd2600e42f54f37aad17d0fdddd51d8/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:36.632502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec5248-cad7-4e92-ab52-8120e6113fb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=2504616647f74ee6b91a718f458eb9a2, user_id=cfd2600e42f54f37aad17d0fdddd51d8, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:36.633919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec5248-cad7-4e92-ab52-8120e6113fb4 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:36.648921 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 547/1097] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:33:36 2022] PUT /identity/v3/projects/2504616647f74ee6b91a718f458eb9a2/users/cfd2600e42f54f37aad17d0fdddd51d8/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:33:36.735020 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2502f-2650-4adb-88ac-1949c19a90cd None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:36.771985 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775d264-1f7f-4778-897f-eb432a5ae2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:36.772565 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775d264-1f7f-4778-897f-eb432a5ae2c9 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:36.772849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775d264-1f7f-4778-897f-eb432a5ae2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:36.773130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775d264-1f7f-4778-897f-eb432a5ae2c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:36.773967 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775d264-1f7f-4778-897f-eb432a5ae2c9 None admin] RBAC: Adding query filter params (role.id=8ce71a4c1e804871b92156359277b9ee, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=479a376dcac14c509c1c55f0046d88cf) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:36.774282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775d264-1f7f-4778-897f-eb432a5ae2c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:36.775787 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775d264-1f7f-4778-897f-eb432a5ae2c9 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:36.783199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 551/1098] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:33:36 2022] GET /identity/v3/role_assignments?user.id=479a376dcac14c509c1c55f0046d88cf&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=8ce71a4c1e804871b92156359277b9ee => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:33:38.666905 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6528efa-e76e-4a91-b98e-9a16e0c73a32 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:38.667525 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6528efa-e76e-4a91-b98e-9a16e0c73a32 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:38.667525 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6528efa-e76e-4a91-b98e-9a16e0c73a32 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:38.668871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 548/1099] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:38.674900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c48734-5e8c-40a7-a4c5-ebc2fd61a398 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:38.675444 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c48734-5e8c-40a7-a4c5-ebc2fd61a398 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:38.675545 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c48734-5e8c-40a7-a4c5-ebc2fd61a398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:38.691914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-83c48734-5e8c-40a7-a4c5-ebc2fd61a398 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:38.693114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-83c48734-5e8c-40a7-a4c5-ebc2fd61a398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:38.749007 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:38.760345 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 552/1100] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:38.785187 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6528efa-e76e-4a91-b98e-9a16e0c73a32 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:38.882894 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:38.894426 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6528efa-e76e-4a91-b98e-9a16e0c73a32 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:38.928646 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c91fa-b958-4db5-898f-a42481a93a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:38.929355 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c91fa-b958-4db5-898f-a42481a93a02 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:38.929634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c91fa-b958-4db5-898f-a42481a93a02 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:38.929904 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c91fa-b958-4db5-898f-a42481a93a02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:38.940570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c91fa-b958-4db5-898f-a42481a93a02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:33:38.948139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-521c91fa-b958-4db5-898f-a42481a93a02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:33:38.948139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:38.948139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:38.948139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:38.948139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:38.948139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:38.948139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:38.948139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:38.948139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:38.948139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:38.948139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:38.948139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:38.948139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:33:38.948139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:33:38.948139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:33:38.948139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:33:38.949285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:33:38.949285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:33:38.949285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:33:38.949285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:33:38.949285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:38.949285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:38.949285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:38.949285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:38.949285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:38.949285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:38.949285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:38.949285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:38.949285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:38.949285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:38.949285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:38.949285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:38.950581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:38.950581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:38.950581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:38.950581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:38.950581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:33:38.950581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:33:38.950581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:33:38.950581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:33:38.950581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:33:38.950581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:33:38.950581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:33:38.950581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:38.950581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 549/1101] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:33:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:33:38.956211 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c48734-5e8c-40a7-a4c5-ebc2fd61a398 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:39.058439 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c48734-5e8c-40a7-a4c5-ebc2fd61a398 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:39.093071 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4edab6-4190-4acb-b5c2-2c5b3a75e585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:39.093741 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4edab6-4190-4acb-b5c2-2c5b3a75e585 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:39.094025 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4edab6-4190-4acb-b5c2-2c5b3a75e585 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:39.094303 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4edab6-4190-4acb-b5c2-2c5b3a75e585 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:39.095670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4edab6-4190-4acb-b5c2-2c5b3a75e585 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:39.095979 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4edab6-4190-4acb-b5c2-2c5b3a75e585 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:39.097350 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4edab6-4190-4acb-b5c2-2c5b3a75e585 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:39.106079 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 553/1102] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:33:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:39.112165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c91fa-b958-4db5-898f-a42481a93a02 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:39.215886 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c91fa-b958-4db5-898f-a42481a93a02 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:39.255213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7a4e8-4f16-4f19-a3c3-535265524c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:39.256200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7a4e8-4f16-4f19-a3c3-535265524c43 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:39.256475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7a4e8-4f16-4f19-a3c3-535265524c43 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:39.256743 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7a4e8-4f16-4f19-a3c3-535265524c43 None admin] PATH_INFO: `/v3/users/cfd2600e42f54f37aad17d0fdddd51d8` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:39.257941 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7a4e8-4f16-4f19-a3c3-535265524c43 None admin] RBAC: Authorizing `identity:get_user(user_id=cfd2600e42f54f37aad17d0fdddd51d8)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:39.259394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7a4e8-4f16-4f19-a3c3-535265524c43 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:39.261063 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 550/1103] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:33:39 2022] GET /identity/v3/users/cfd2600e42f54f37aad17d0fdddd51d8 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:39.268646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4edab6-4190-4acb-b5c2-2c5b3a75e585 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:39.323760 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:39.368476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4edab6-4190-4acb-b5c2-2c5b3a75e585 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:39.405056 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890a217-15d2-456c-9c2e-628b8525f0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:39.405705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890a217-15d2-456c-9c2e-628b8525f0d2 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:39.405988 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890a217-15d2-456c-9c2e-628b8525f0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:39.406267 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890a217-15d2-456c-9c2e-628b8525f0d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:39.475956 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890a217-15d2-456c-9c2e-628b8525f0d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:39.477335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890a217-15d2-456c-9c2e-628b8525f0d2 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:39.534742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:39.582947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 554/1104] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:33:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 22 11:33:39.588897 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7a4e8-4f16-4f19-a3c3-535265524c43 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:39.622646 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:39.703235 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7a4e8-4f16-4f19-a3c3-535265524c43 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:39.739922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea8dd0-f1df-4bf4-9f6d-e4bbe3bcf4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:39.740694 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea8dd0-f1df-4bf4-9f6d-e4bbe3bcf4d6 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:39.740977 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea8dd0-f1df-4bf4-9f6d-e4bbe3bcf4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:39.741256 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea8dd0-f1df-4bf4-9f6d-e4bbe3bcf4d6 None admin] PATH_INFO: `/v3/projects/2504616647f74ee6b91a718f458eb9a2` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:39.742484 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea8dd0-f1df-4bf4-9f6d-e4bbe3bcf4d6 None admin] RBAC: Authorizing `identity:get_project(project_id=2504616647f74ee6b91a718f458eb9a2)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:39.743979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea8dd0-f1df-4bf4-9f6d-e4bbe3bcf4d6 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:39.745746 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 551/1105] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:33:39 2022] GET /identity/v3/projects/2504616647f74ee6b91a718f458eb9a2 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:39.752515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890a217-15d2-456c-9c2e-628b8525f0d2 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:39.854691 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890a217-15d2-456c-9c2e-628b8525f0d2 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:39.892293 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab066b-68e5-440e-bfd7-ed60a0a5f43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:39.892960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab066b-68e5-440e-bfd7-ed60a0a5f43b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:39.893245 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab066b-68e5-440e-bfd7-ed60a0a5f43b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:39.893520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab066b-68e5-440e-bfd7-ed60a0a5f43b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:39.894358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab066b-68e5-440e-bfd7-ed60a0a5f43b None admin] RBAC: Adding query filter params (role.id=96c584428d924fdba9671949f760c406, scope.project.id=2504616647f74ee6b91a718f458eb9a2, user.id=cfd2600e42f54f37aad17d0fdddd51d8) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:39.894666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab066b-68e5-440e-bfd7-ed60a0a5f43b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:39.896230 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab066b-68e5-440e-bfd7-ed60a0a5f43b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:39.902933 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 555/1106] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:33:39 2022] GET /identity/v3/role_assignments?user.id=cfd2600e42f54f37aad17d0fdddd51d8&scope.project.id=2504616647f74ee6b91a718f458eb9a2&role.id=96c584428d924fdba9671949f760c406 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:41.996232 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17977269-e20e-4bfe-bfaf-59f7d9e2920a None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:41.996779 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17977269-e20e-4bfe-bfaf-59f7d9e2920a None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:41.996779 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17977269-e20e-4bfe-bfaf-59f7d9e2920a None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:41.998169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 552/1107] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:42.004949 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cef9c-1a7d-4b91-adb6-32d87ee6c0db None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:42.005513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cef9c-1a7d-4b91-adb6-32d87ee6c0db None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:42.005513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66cef9c-1a7d-4b91-adb6-32d87ee6c0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:42.022938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-e66cef9c-1a7d-4b91-adb6-32d87ee6c0db None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:42.024233 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e66cef9c-1a7d-4b91-adb6-32d87ee6c0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:42.039776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:42.099959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 556/1108] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:42.136460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17977269-e20e-4bfe-bfaf-59f7d9e2920a None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:42.228646 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:42.242315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17977269-e20e-4bfe-bfaf-59f7d9e2920a None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:42.275220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86de175-c167-4192-b228-31552e22565a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:42.276039 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86de175-c167-4192-b228-31552e22565a None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:42.276324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86de175-c167-4192-b228-31552e22565a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:42.276599 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86de175-c167-4192-b228-31552e22565a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:42.277771 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86de175-c167-4192-b228-31552e22565a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:42.279198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86de175-c167-4192-b228-31552e22565a None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:42.280851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 553/1109] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:42.287240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cef9c-1a7d-4b91-adb6-32d87ee6c0db None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:42.383899 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66cef9c-1a7d-4b91-adb6-32d87ee6c0db None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:42.416941 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9646e73-66e7-4c90-8455-8fbc4bfca197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:42.417608 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9646e73-66e7-4c90-8455-8fbc4bfca197 None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:42.417893 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9646e73-66e7-4c90-8455-8fbc4bfca197 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:42.418170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9646e73-66e7-4c90-8455-8fbc4bfca197 None admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:42.418888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9646e73-66e7-4c90-8455-8fbc4bfca197 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:42.420362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9646e73-66e7-4c90-8455-8fbc4bfca197 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:42.440275 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 557/1110] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jun 22 11:33:42 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:33:44.526922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29dcc70-485d-458f-b780-f881e9cc5fe3 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:44.527503 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29dcc70-485d-458f-b780-f881e9cc5fe3 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:44.527503 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29dcc70-485d-458f-b780-f881e9cc5fe3 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:44.528871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 554/1111] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:44.534630 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a988c7-261a-42b3-9cf9-2e709691a179 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:44.535164 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a988c7-261a-42b3-9cf9-2e709691a179 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:44.535164 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a988c7-261a-42b3-9cf9-2e709691a179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:44.541351 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:44.552884 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-10a988c7-261a-42b3-9cf9-2e709691a179 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:44.554083 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-10a988c7-261a-42b3-9cf9-2e709691a179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:44.625249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 558/1112] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:44.650527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29dcc70-485d-458f-b780-f881e9cc5fe3 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:44.760507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29dcc70-485d-458f-b780-f881e9cc5fe3 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:44.795869 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae07866-3601-4271-80ef-8fb598ac166f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:44.796627 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae07866-3601-4271-80ef-8fb598ac166f None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:44.796907 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae07866-3601-4271-80ef-8fb598ac166f None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:44.797179 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae07866-3601-4271-80ef-8fb598ac166f None admin] PATH_INFO: `/v3/roles/b4be7021e4104740b8b0b3b40ecd3d59` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:44.798632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae07866-3601-4271-80ef-8fb598ac166f None admin] RBAC: Authorizing `identity:get_role(role_id=b4be7021e4104740b8b0b3b40ecd3d59)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:44.800024 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae07866-3601-4271-80ef-8fb598ac166f None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:44.801417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 555/1113] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:33:44 2022] GET /identity/v3/roles/b4be7021e4104740b8b0b3b40ecd3d59 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:44.807549 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a988c7-261a-42b3-9cf9-2e709691a179 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:44.908641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a988c7-261a-42b3-9cf9-2e709691a179 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:44.944267 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e6302-9a7a-4aa3-8035-acb50d5801d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:44.945091 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e6302-9a7a-4aa3-8035-acb50d5801d3 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:44.945359 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e6302-9a7a-4aa3-8035-acb50d5801d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:44.945627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e6302-9a7a-4aa3-8035-acb50d5801d3 None admin] PATH_INFO: `/v3/users/d188a0a9871b4f449385fc020189a6c7` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:44.947869 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:44.962085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e6302-9a7a-4aa3-8035-acb50d5801d3 None admin] RBAC: Authorizing `identity:get_user(user_id=d188a0a9871b4f449385fc020189a6c7)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:44.963585 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e6302-9a7a-4aa3-8035-acb50d5801d3 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:44.965335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 559/1114] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:33:44 2022] GET /identity/v3/users/d188a0a9871b4f449385fc020189a6c7 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:44.972029 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae07866-3601-4271-80ef-8fb598ac166f None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:44.998700 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:45.079865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae07866-3601-4271-80ef-8fb598ac166f None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:45.113904 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69309c-8732-4c10-b9c4-a3eafc7355a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:45.114549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69309c-8732-4c10-b9c4-a3eafc7355a0 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:45.114828 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69309c-8732-4c10-b9c4-a3eafc7355a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:45.115105 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69309c-8732-4c10-b9c4-a3eafc7355a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:45.184780 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69309c-8732-4c10-b9c4-a3eafc7355a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:45.186222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69309c-8732-4c10-b9c4-a3eafc7355a0 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:45.211846 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:45.291180 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 556/1115] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:33:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 22 11:33:45.297443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e6302-9a7a-4aa3-8035-acb50d5801d3 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:45.407762 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e6302-9a7a-4aa3-8035-acb50d5801d3 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:45.456139 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab6b86-36ef-4f01-842c-6fdbd5468274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:45.457657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab6b86-36ef-4f01-842c-6fdbd5468274 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:45.458169 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab6b86-36ef-4f01-842c-6fdbd5468274 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:45.458677 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab6b86-36ef-4f01-842c-6fdbd5468274 None admin] PATH_INFO: `/v3/projects/2504616647f74ee6b91a718f458eb9a2` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:45.460892 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab6b86-36ef-4f01-842c-6fdbd5468274 None admin] RBAC: Authorizing `identity:get_project(project_id=2504616647f74ee6b91a718f458eb9a2)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:45.463705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab6b86-36ef-4f01-842c-6fdbd5468274 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:45.467037 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 560/1116] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:33:45 2022] GET /identity/v3/projects/2504616647f74ee6b91a718f458eb9a2 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:45.473876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69309c-8732-4c10-b9c4-a3eafc7355a0 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:45.582644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69309c-8732-4c10-b9c4-a3eafc7355a0 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:45.608269 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:45.619437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9dd5d-b08b-4cb0-b75d-246aba76eeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:45.620099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9dd5d-b08b-4cb0-b75d-246aba76eeb1 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:45.620379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9dd5d-b08b-4cb0-b75d-246aba76eeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:45.620654 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9dd5d-b08b-4cb0-b75d-246aba76eeb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:45.621492 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9dd5d-b08b-4cb0-b75d-246aba76eeb1 None admin] RBAC: Adding query filter params (role.id=b4be7021e4104740b8b0b3b40ecd3d59, scope.project.id=2504616647f74ee6b91a718f458eb9a2, user.id=d188a0a9871b4f449385fc020189a6c7) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:45.621804 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9dd5d-b08b-4cb0-b75d-246aba76eeb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:45.623250 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9dd5d-b08b-4cb0-b75d-246aba76eeb1 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:45.629593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 557/1117] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:33:45 2022] GET /identity/v3/role_assignments?user.id=d188a0a9871b4f449385fc020189a6c7&scope.project.id=2504616647f74ee6b91a718f458eb9a2&role.id=b4be7021e4104740b8b0b3b40ecd3d59 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:47.688613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeab6ad-abca-4bcc-9d6e-b8a46b3c6572 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:47.689418 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeab6ad-abca-4bcc-9d6e-b8a46b3c6572 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:47.689418 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeab6ad-abca-4bcc-9d6e-b8a46b3c6572 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:47.690511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 561/1118] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:47.696443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2ed01-7869-44dd-b611-4e920cc9105d None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:47.697078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2ed01-7869-44dd-b611-4e920cc9105d None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:47.697078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2ed01-7869-44dd-b611-4e920cc9105d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:47.718510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-f5e2ed01-7869-44dd-b611-4e920cc9105d None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:47.720622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f5e2ed01-7869-44dd-b611-4e920cc9105d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:47.790928 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 558/1119] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:47.814960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeab6ad-abca-4bcc-9d6e-b8a46b3c6572 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:47.877932 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:47.922202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeab6ad-abca-4bcc-9d6e-b8a46b3c6572 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:47.958858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff84946-02fc-4713-9521-6485251b3cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:47.959822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff84946-02fc-4713-9521-6485251b3cf8 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:47.960146 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff84946-02fc-4713-9521-6485251b3cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:47.960468 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff84946-02fc-4713-9521-6485251b3cf8 None admin] PATH_INFO: `/v3/roles/b4be7021e4104740b8b0b3b40ecd3d59` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:47.962051 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff84946-02fc-4713-9521-6485251b3cf8 None admin] RBAC: Authorizing `identity:get_role(role_id=b4be7021e4104740b8b0b3b40ecd3d59)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:47.963538 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff84946-02fc-4713-9521-6485251b3cf8 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:47.964979 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 562/1120] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:33:47 2022] GET /identity/v3/roles/b4be7021e4104740b8b0b3b40ecd3d59 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:47.971311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2ed01-7869-44dd-b611-4e920cc9105d None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:48.069782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2ed01-7869-44dd-b611-4e920cc9105d None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:48.087107 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:48.105798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09ad7b-3791-4424-99d3-ce899cb441fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:48.106661 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09ad7b-3791-4424-99d3-ce899cb441fa None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:48.106939 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09ad7b-3791-4424-99d3-ce899cb441fa None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:48.107215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09ad7b-3791-4424-99d3-ce899cb441fa None admin] PATH_INFO: `/v3/users/d188a0a9871b4f449385fc020189a6c7` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:48.108434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09ad7b-3791-4424-99d3-ce899cb441fa None admin] RBAC: Authorizing `identity:get_user(user_id=d188a0a9871b4f449385fc020189a6c7)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:48.109884 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09ad7b-3791-4424-99d3-ce899cb441fa None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:48.112067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 559/1121] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:33:47 2022] GET /identity/v3/users/d188a0a9871b4f449385fc020189a6c7 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:48.118618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff84946-02fc-4713-9521-6485251b3cf8 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:48.219227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff84946-02fc-4713-9521-6485251b3cf8 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:48.247086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:48.255943 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f5d11-e571-4300-9545-d81e0415f22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:48.256810 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f5d11-e571-4300-9545-d81e0415f22f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:48.257091 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f5d11-e571-4300-9545-d81e0415f22f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:48.257369 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f5d11-e571-4300-9545-d81e0415f22f None admin] PATH_INFO: `/v3/projects/2504616647f74ee6b91a718f458eb9a2` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:48.258556 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f5d11-e571-4300-9545-d81e0415f22f None admin] RBAC: Authorizing `identity:get_project(project_id=2504616647f74ee6b91a718f458eb9a2)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:48.260058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f5d11-e571-4300-9545-d81e0415f22f None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:48.261811 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 563/1122] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:33:48 2022] GET /identity/v3/projects/2504616647f74ee6b91a718f458eb9a2 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:48.268928 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09ad7b-3791-4424-99d3-ce899cb441fa None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:48.370623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09ad7b-3791-4424-99d3-ce899cb441fa None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:48.407943 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d92dde-46dc-4453-b47e-36a50a76320d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:48.409304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d92dde-46dc-4453-b47e-36a50a76320d None admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:48.409715 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d92dde-46dc-4453-b47e-36a50a76320d None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:48.410148 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d92dde-46dc-4453-b47e-36a50a76320d None admin] PATH_INFO: `/v3/projects/2504616647f74ee6b91a718f458eb9a2/users/d188a0a9871b4f449385fc020189a6c7/roles/b4be7021e4104740b8b0b3b40ecd3d59` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:48.412477 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d92dde-46dc-4453-b47e-36a50a76320d None admin] RBAC: Authorizing `identity:create_grant(project_id=2504616647f74ee6b91a718f458eb9a2, user_id=d188a0a9871b4f449385fc020189a6c7, role_id=b4be7021e4104740b8b0b3b40ecd3d59)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:48.413925 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d92dde-46dc-4453-b47e-36a50a76320d None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:48.427910 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 560/1123] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:33:48 2022] PUT /identity/v3/projects/2504616647f74ee6b91a718f458eb9a2/users/d188a0a9871b4f449385fc020189a6c7/roles/b4be7021e4104740b8b0b3b40ecd3d59 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:33:50.474853 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a52b5-f1f3-464e-a3b1-c417ebc334df None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:50.475413 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a52b5-f1f3-464e-a3b1-c417ebc334df None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:50.475413 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a52b5-f1f3-464e-a3b1-c417ebc334df None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:50.476768 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 564/1124] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:50.482460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26885c11-aefa-48a6-8f24-17d7d1f855e9 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:50.482946 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26885c11-aefa-48a6-8f24-17d7d1f855e9 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:50.483040 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26885c11-aefa-48a6-8f24-17d7d1f855e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:50.499973 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-26885c11-aefa-48a6-8f24-17d7d1f855e9 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:50.501702 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-26885c11-aefa-48a6-8f24-17d7d1f855e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:50.546991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:50.575058 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 561/1125] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:33:50.599884 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a52b5-f1f3-464e-a3b1-c417ebc334df None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:50.705150 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a52b5-f1f3-464e-a3b1-c417ebc334df None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:50.741990 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeebbea-cc31-42f1-b876-43782573ba11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:50.742701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeebbea-cc31-42f1-b876-43782573ba11 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:50.742979 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeebbea-cc31-42f1-b876-43782573ba11 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:50.743247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeebbea-cc31-42f1-b876-43782573ba11 None admin] PATH_INFO: `/v3/roles/b4be7021e4104740b8b0b3b40ecd3d59` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:50.744957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeebbea-cc31-42f1-b876-43782573ba11 None admin] RBAC: Authorizing `identity:get_role(role_id=b4be7021e4104740b8b0b3b40ecd3d59)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:50.746339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeebbea-cc31-42f1-b876-43782573ba11 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:50.747771 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 565/1126] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:33:50 2022] GET /identity/v3/roles/b4be7021e4104740b8b0b3b40ecd3d59 => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:50.754132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26885c11-aefa-48a6-8f24-17d7d1f855e9 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:50.858690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26885c11-aefa-48a6-8f24-17d7d1f855e9 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:50.895664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f812698-ddae-4ef9-895b-f89c5e7ce47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:50.896538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f812698-ddae-4ef9-895b-f89c5e7ce47a None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:50.896816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f812698-ddae-4ef9-895b-f89c5e7ce47a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:50.897098 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f812698-ddae-4ef9-895b-f89c5e7ce47a None admin] PATH_INFO: `/v3/users/d188a0a9871b4f449385fc020189a6c7` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:50.898332 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f812698-ddae-4ef9-895b-f89c5e7ce47a None admin] RBAC: Authorizing `identity:get_user(user_id=d188a0a9871b4f449385fc020189a6c7)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:50.899813 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f812698-ddae-4ef9-895b-f89c5e7ce47a None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:50.901505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 562/1127] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:33:50 2022] GET /identity/v3/users/d188a0a9871b4f449385fc020189a6c7 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:50.907594 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeebbea-cc31-42f1-b876-43782573ba11 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:50.973071 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:51.011768 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeebbea-cc31-42f1-b876-43782573ba11 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:51.046319 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ae2e0-9d3e-4dae-a510-36d3e68a7b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:51.046981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ae2e0-9d3e-4dae-a510-36d3e68a7b10 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:51.047270 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ae2e0-9d3e-4dae-a510-36d3e68a7b10 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:51.048138 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ae2e0-9d3e-4dae-a510-36d3e68a7b10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:51.122190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ae2e0-9d3e-4dae-a510-36d3e68a7b10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:51.123800 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ae2e0-9d3e-4dae-a510-36d3e68a7b10 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:51.191411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:51.235977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 566/1128] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:33:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 22 11:33:51.242215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f812698-ddae-4ef9-895b-f89c5e7ce47a None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:51.292366 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:51.352161 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f812698-ddae-4ef9-895b-f89c5e7ce47a None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:51.388612 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229dfae-75a9-4852-9883-aab41015f5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:51.389817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229dfae-75a9-4852-9883-aab41015f5df None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:51.390138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229dfae-75a9-4852-9883-aab41015f5df None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:51.390455 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229dfae-75a9-4852-9883-aab41015f5df None admin] PATH_INFO: `/v3/projects/2504616647f74ee6b91a718f458eb9a2` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:51.392349 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229dfae-75a9-4852-9883-aab41015f5df None admin] RBAC: Authorizing `identity:get_project(project_id=2504616647f74ee6b91a718f458eb9a2)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:51.393849 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229dfae-75a9-4852-9883-aab41015f5df None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:51.395656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 563/1129] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:33:51 2022] GET /identity/v3/projects/2504616647f74ee6b91a718f458eb9a2 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:51.404717 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ae2e0-9d3e-4dae-a510-36d3e68a7b10 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:51.567297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ae2e0-9d3e-4dae-a510-36d3e68a7b10 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:51.608424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868ae48-615c-4046-9e50-2857a13a9d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:51.609094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868ae48-615c-4046-9e50-2857a13a9d24 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:51.609379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868ae48-615c-4046-9e50-2857a13a9d24 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:51.609657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868ae48-615c-4046-9e50-2857a13a9d24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:51.611069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868ae48-615c-4046-9e50-2857a13a9d24 None admin] RBAC: Adding query filter params (role.id=b4be7021e4104740b8b0b3b40ecd3d59, scope.project.id=2504616647f74ee6b91a718f458eb9a2, user.id=d188a0a9871b4f449385fc020189a6c7) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:51.611407 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868ae48-615c-4046-9e50-2857a13a9d24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:51.612849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868ae48-615c-4046-9e50-2857a13a9d24 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:51.618971 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 567/1130] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:33:51 2022] GET /identity/v3/role_assignments?user.id=d188a0a9871b4f449385fc020189a6c7&scope.project.id=2504616647f74ee6b91a718f458eb9a2&role.id=b4be7021e4104740b8b0b3b40ecd3d59 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:53.755662 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4251b1-36fc-48d8-849a-6e7778801692 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:53.756262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4251b1-36fc-48d8-849a-6e7778801692 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:53.756262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4251b1-36fc-48d8-849a-6e7778801692 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:53.757604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 564/1131] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:53.763244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b20396-0c1d-4350-b3cc-e978a4837d94 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:53.763788 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b20396-0c1d-4350-b3cc-e978a4837d94 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:53.763892 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b20396-0c1d-4350-b3cc-e978a4837d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:53.782263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-c6b20396-0c1d-4350-b3cc-e978a4837d94 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:53.783526 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-c6b20396-0c1d-4350-b3cc-e978a4837d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:53.809794 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:53.865833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 568/1132] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:53.890397 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4251b1-36fc-48d8-849a-6e7778801692 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:54.001473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4251b1-36fc-48d8-849a-6e7778801692 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:54.015592 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:54.037273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55532fbd-3158-495b-8e4c-e6511d932048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:54.038078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55532fbd-3158-495b-8e4c-e6511d932048 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:54.038357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55532fbd-3158-495b-8e4c-e6511d932048 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:54.038627 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55532fbd-3158-495b-8e4c-e6511d932048 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:54.039916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55532fbd-3158-495b-8e4c-e6511d932048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:54.041370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55532fbd-3158-495b-8e4c-e6511d932048 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:54.043056 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 565/1133] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:54.050058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b20396-0c1d-4350-b3cc-e978a4837d94 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:54.158883 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b20396-0c1d-4350-b3cc-e978a4837d94 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:54.197373 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f14aeb-0e73-4745-95b3-4c79728af0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:54.198086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f14aeb-0e73-4745-95b3-4c79728af0c7 None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:54.198378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f14aeb-0e73-4745-95b3-4c79728af0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:54.198657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f14aeb-0e73-4745-95b3-4c79728af0c7 None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:54.199411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f14aeb-0e73-4745-95b3-4c79728af0c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:54.200874 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f14aeb-0e73-4745-95b3-4c79728af0c7 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:54.230850 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 569/1134] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jun 22 11:33:54 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:33:56.396859 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7730d61-4a7e-44b4-b3a3-31a3120e9693 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:56.397400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7730d61-4a7e-44b4-b3a3-31a3120e9693 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:56.397400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7730d61-4a7e-44b4-b3a3-31a3120e9693 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:56.398788 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 566/1135] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:56.404500 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5541c-d9af-4d30-b43e-ceb27748bbd9 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:56.404922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5541c-d9af-4d30-b43e-ceb27748bbd9 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:56.405061 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5541c-d9af-4d30-b43e-ceb27748bbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:56.421637 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-1ed5541c-d9af-4d30-b43e-ceb27748bbd9 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:56.423446 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-1ed5541c-d9af-4d30-b43e-ceb27748bbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:56.426346 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:56.493006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 570/1136] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:56.518949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7730d61-4a7e-44b4-b3a3-31a3120e9693 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:56.628423 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7730d61-4a7e-44b4-b3a3-31a3120e9693 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:56.663387 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97acc1c-218d-495c-995b-aac49ce34754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:56.664235 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97acc1c-218d-495c-995b-aac49ce34754 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:56.664519 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97acc1c-218d-495c-995b-aac49ce34754 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:56.664798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97acc1c-218d-495c-995b-aac49ce34754 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:56.665993 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97acc1c-218d-495c-995b-aac49ce34754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:56.667450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97acc1c-218d-495c-995b-aac49ce34754 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:56.669105 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 567/1137] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:33:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:56.675906 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5541c-d9af-4d30-b43e-ceb27748bbd9 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:56.770466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5541c-d9af-4d30-b43e-ceb27748bbd9 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:56.803368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831dd43-7955-4be9-acdb-033ffd9033b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:56.804042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831dd43-7955-4be9-acdb-033ffd9033b0 None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:56.804323 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831dd43-7955-4be9-acdb-033ffd9033b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:56.804597 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831dd43-7955-4be9-acdb-033ffd9033b0 None admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:56.805289 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831dd43-7955-4be9-acdb-033ffd9033b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:56.806722 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831dd43-7955-4be9-acdb-033ffd9033b0 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:56.816279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:56.826600 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 571/1138] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jun 22 11:33:56 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:33:58.888947 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85850c7a-82a1-4c32-96e3-f41021457fdc None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:58.889757 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85850c7a-82a1-4c32-96e3-f41021457fdc None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:58.889757 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85850c7a-82a1-4c32-96e3-f41021457fdc None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:58.891148 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 568/1139] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:33:58.896294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2f24a-942d-45c5-8e30-4ef6c309d844 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:58.896665 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2f24a-942d-45c5-8e30-4ef6c309d844 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:58.896890 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2f24a-942d-45c5-8e30-4ef6c309d844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:58.914398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-3bd2f24a-942d-45c5-8e30-4ef6c309d844 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:33:58.915611 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-3bd2f24a-942d-45c5-8e30-4ef6c309d844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:33:58.982132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 572/1140] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:33:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:33:59.006043 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85850c7a-82a1-4c32-96e3-f41021457fdc None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:59.055522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:59.106020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85850c7a-82a1-4c32-96e3-f41021457fdc None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:59.139821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717dda8-9c50-4180-8979-8038a5a36e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:59.140515 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717dda8-9c50-4180-8979-8038a5a36e18 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:59.140795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717dda8-9c50-4180-8979-8038a5a36e18 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:59.141716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717dda8-9c50-4180-8979-8038a5a36e18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:59.151986 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717dda8-9c50-4180-8979-8038a5a36e18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:33:59.160170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-a717dda8-9c50-4180-8979-8038a5a36e18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:33:59.160170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:33:59.160170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:33:59.160170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:33:59.160170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:33:59.160170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:33:59.160170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:33:59.160170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:33:59.160170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:33:59.160170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:33:59.160170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:33:59.160170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:33:59.160170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:33:59.160170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:33:59.160170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:33:59.160170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:33:59.161302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:33:59.161302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:33:59.161302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:33:59.161302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:33:59.161302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:33:59.161302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:33:59.161302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:33:59.161302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:33:59.161302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:33:59.161302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:33:59.161302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:33:59.161302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:33:59.161302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:33:59.161302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:33:59.161302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:33:59.161302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:33:59.167186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:33:59.167186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:33:59.167186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:33:59.167186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:33:59.167186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:33:59.167186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:33:59.167186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:33:59.167186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:33:59.167186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:33:59.167186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:33:59.167186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:33:59.167186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:33:59.167186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 569/1141] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:33:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:33:59.168068 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2f24a-942d-45c5-8e30-4ef6c309d844 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:59.271935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2f24a-942d-45c5-8e30-4ef6c309d844 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:59.299797 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:59.308723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e81eff-1a70-4013-9923-0b00c0b121dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:59.309401 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e81eff-1a70-4013-9923-0b00c0b121dc None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:59.309695 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e81eff-1a70-4013-9923-0b00c0b121dc None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:59.309970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e81eff-1a70-4013-9923-0b00c0b121dc None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:59.310729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e81eff-1a70-4013-9923-0b00c0b121dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:33:59.311043 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e81eff-1a70-4013-9923-0b00c0b121dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:59.312473 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e81eff-1a70-4013-9923-0b00c0b121dc None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:59.321568 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 573/1142] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:33:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:59.331136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717dda8-9c50-4180-8979-8038a5a36e18 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:59.429967 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717dda8-9c50-4180-8979-8038a5a36e18 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:59.438386 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:59.463593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbad2ef-a5e5-4043-867c-be7a14b5a32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:59.464458 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbad2ef-a5e5-4043-867c-be7a14b5a32e None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:59.464744 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbad2ef-a5e5-4043-867c-be7a14b5a32e None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:59.465024 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbad2ef-a5e5-4043-867c-be7a14b5a32e None admin] PATH_INFO: `/v3/users/75991f1afee34530b764244fa8e2d6e9` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:59.479024 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbad2ef-a5e5-4043-867c-be7a14b5a32e None admin] RBAC: Authorizing `identity:get_user(user_id=75991f1afee34530b764244fa8e2d6e9)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:59.480574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbad2ef-a5e5-4043-867c-be7a14b5a32e None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:59.482214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 570/1143] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:33:59 2022] GET /identity/v3/users/75991f1afee34530b764244fa8e2d6e9 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:59.488033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e81eff-1a70-4013-9923-0b00c0b121dc None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:59.600813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e81eff-1a70-4013-9923-0b00c0b121dc None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:59.635065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c011d-c925-4d45-93b0-50349c6a966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:59.635772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c011d-c925-4d45-93b0-50349c6a966b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:59.636059 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c011d-c925-4d45-93b0-50349c6a966b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:59.636338 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c011d-c925-4d45-93b0-50349c6a966b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:59.691060 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:33:59.703538 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c011d-c925-4d45-93b0-50349c6a966b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:59.705486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c011d-c925-4d45-93b0-50349c6a966b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:59.810735 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 574/1144] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:33:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 22 11:33:59.813738 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbad2ef-a5e5-4043-867c-be7a14b5a32e None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:33:59.936356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbad2ef-a5e5-4043-867c-be7a14b5a32e None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:33:59.971523 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759b9be-0961-455f-887e-8189a0c77cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:33:59.972369 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759b9be-0961-455f-887e-8189a0c77cf3 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:33:59.972646 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759b9be-0961-455f-887e-8189a0c77cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:33:59.972915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759b9be-0961-455f-887e-8189a0c77cf3 None admin] PATH_INFO: `/v3/projects/19f1cc7ea0fa456f9dab19a4039de405` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:33:59.974077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759b9be-0961-455f-887e-8189a0c77cf3 None admin] RBAC: Authorizing `identity:get_project(project_id=19f1cc7ea0fa456f9dab19a4039de405)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:33:59.975525 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759b9be-0961-455f-887e-8189a0c77cf3 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:33:59.977215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 571/1145] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:33:59 2022] GET /identity/v3/projects/19f1cc7ea0fa456f9dab19a4039de405 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:33:59.983325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c011d-c925-4d45-93b0-50349c6a966b None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:00.084132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:00.096716 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c011d-c925-4d45-93b0-50349c6a966b None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:00.129047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efc73c-bfc9-4439-8ccf-73da8d182cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:00.129683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efc73c-bfc9-4439-8ccf-73da8d182cce None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:00.129963 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efc73c-bfc9-4439-8ccf-73da8d182cce None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:00.130237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efc73c-bfc9-4439-8ccf-73da8d182cce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:00.131059 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1efc73c-bfc9-4439-8ccf-73da8d182cce None admin] RBAC: Adding query filter params (role.id=96c584428d924fdba9671949f760c406, scope.project.id=19f1cc7ea0fa456f9dab19a4039de405, user.id=75991f1afee34530b764244fa8e2d6e9) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:34:00.131390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1efc73c-bfc9-4439-8ccf-73da8d182cce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:00.132813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1efc73c-bfc9-4439-8ccf-73da8d182cce None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:00.138041 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 575/1146] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:33:59 2022] GET /identity/v3/role_assignments?user.id=75991f1afee34530b764244fa8e2d6e9&scope.project.id=19f1cc7ea0fa456f9dab19a4039de405&role.id=96c584428d924fdba9671949f760c406 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:02.199185 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ed030-34de-4c71-aa24-09bfb47b6ee0 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:02.200071 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ed030-34de-4c71-aa24-09bfb47b6ee0 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:02.200371 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ed030-34de-4c71-aa24-09bfb47b6ee0 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:02.201791 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 572/1147] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:34:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:34:02.207516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ef942-7466-43e7-abbe-e447897e328b None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:02.207901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ef942-7466-43e7-abbe-e447897e328b None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:02.208087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ef942-7466-43e7-abbe-e447897e328b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:02.225291 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-ea4ef942-7466-43e7-abbe-e447897e328b None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:34:02.226494 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ea4ef942-7466-43e7-abbe-e447897e328b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:34:02.291732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 576/1148] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:34:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:34:02.314382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ed030-34de-4c71-aa24-09bfb47b6ee0 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:02.349060 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:02.414296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ed030-34de-4c71-aa24-09bfb47b6ee0 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:02.446948 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036b9b7-223e-4767-b5f1-0230da211559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:02.447761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036b9b7-223e-4767-b5f1-0230da211559 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:02.448047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036b9b7-223e-4767-b5f1-0230da211559 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:02.448324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036b9b7-223e-4767-b5f1-0230da211559 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:02.459247 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036b9b7-223e-4767-b5f1-0230da211559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:34:02.466755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-a036b9b7-223e-4767-b5f1-0230da211559 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:34:02.466755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:34:02.466755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:34:02.466755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:34:02.466755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:34:02.466755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:34:02.466755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:34:02.466755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:34:02.466755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:34:02.466755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:34:02.466755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:34:02.466755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:34:02.466755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:34:02.466755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:34:02.466755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:34:02.466755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:34:02.467854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:34:02.467854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:34:02.467854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:34:02.467854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:34:02.467854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:34:02.467854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:34:02.467854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:34:02.467854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:34:02.467854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:34:02.467854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:34:02.467854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:34:02.467854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:34:02.467854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:34:02.467854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:34:02.467854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:34:02.467854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:34:02.469640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:34:02.469640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:34:02.469640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:34:02.469640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:34:02.469640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:34:02.469640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:34:02.469640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:34:02.469640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:34:02.469640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:34:02.469640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:34:02.469640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:34:02.469640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:34:02.469640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 573/1149] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:34:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:34:02.474791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ef942-7466-43e7-abbe-e447897e328b None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:02.560420 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:02.579282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ef942-7466-43e7-abbe-e447897e328b None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:02.612558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ec70d-6d8b-4bcf-a8fe-55053a064107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:02.613237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ec70d-6d8b-4bcf-a8fe-55053a064107 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:02.613523 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ec70d-6d8b-4bcf-a8fe-55053a064107 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:02.613803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ec70d-6d8b-4bcf-a8fe-55053a064107 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:02.614555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ec70d-6d8b-4bcf-a8fe-55053a064107 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:34:02.614865 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ec70d-6d8b-4bcf-a8fe-55053a064107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:02.616276 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ec70d-6d8b-4bcf-a8fe-55053a064107 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:02.624722 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 577/1150] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:34:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:02.631165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036b9b7-223e-4767-b5f1-0230da211559 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:02.738511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:02.746115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036b9b7-223e-4767-b5f1-0230da211559 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:02.783412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1a010-13f8-4a2b-a2d2-314208958e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:02.784289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1a010-13f8-4a2b-a2d2-314208958e6a None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:02.784573 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1a010-13f8-4a2b-a2d2-314208958e6a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:02.784851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1a010-13f8-4a2b-a2d2-314208958e6a None admin] PATH_INFO: `/v3/users/75991f1afee34530b764244fa8e2d6e9` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:02.786053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1a010-13f8-4a2b-a2d2-314208958e6a None admin] RBAC: Authorizing `identity:get_user(user_id=75991f1afee34530b764244fa8e2d6e9)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:02.787517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1a010-13f8-4a2b-a2d2-314208958e6a None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:02.789195 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 574/1151] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:34:02 2022] GET /identity/v3/users/75991f1afee34530b764244fa8e2d6e9 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:02.795059 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ec70d-6d8b-4bcf-a8fe-55053a064107 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:02.896985 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ec70d-6d8b-4bcf-a8fe-55053a064107 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:02.929962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa733b50-1e7f-47ea-a257-8b05d9eecf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:02.930827 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa733b50-1e7f-47ea-a257-8b05d9eecf60 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:02.931108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa733b50-1e7f-47ea-a257-8b05d9eecf60 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:02.931409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa733b50-1e7f-47ea-a257-8b05d9eecf60 None admin] PATH_INFO: `/v3/projects/19f1cc7ea0fa456f9dab19a4039de405` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:02.932611 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa733b50-1e7f-47ea-a257-8b05d9eecf60 None admin] RBAC: Authorizing `identity:get_project(project_id=19f1cc7ea0fa456f9dab19a4039de405)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:02.934044 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa733b50-1e7f-47ea-a257-8b05d9eecf60 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:02.935790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 578/1152] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:34:02 2022] GET /identity/v3/projects/19f1cc7ea0fa456f9dab19a4039de405 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:02.942614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1a010-13f8-4a2b-a2d2-314208958e6a None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:03.045248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1a010-13f8-4a2b-a2d2-314208958e6a None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:03.079650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf5992-e03f-4458-b257-da133a931222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:03.080537 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf5992-e03f-4458-b257-da133a931222 None admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:03.080817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf5992-e03f-4458-b257-da133a931222 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:03.081093 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf5992-e03f-4458-b257-da133a931222 None admin] PATH_INFO: `/v3/projects/19f1cc7ea0fa456f9dab19a4039de405/users/75991f1afee34530b764244fa8e2d6e9/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:03.083038 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf5992-e03f-4458-b257-da133a931222 None admin] RBAC: Authorizing `identity:create_grant(project_id=19f1cc7ea0fa456f9dab19a4039de405, user_id=75991f1afee34530b764244fa8e2d6e9, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:03.084505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf5992-e03f-4458-b257-da133a931222 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:03.098223 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 575/1153] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:34:02 2022] PUT /identity/v3/projects/19f1cc7ea0fa456f9dab19a4039de405/users/75991f1afee34530b764244fa8e2d6e9/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:34:05.141206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54b5a9-06ca-42ef-90d1-e1ec3c2e409f None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:05.141735 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54b5a9-06ca-42ef-90d1-e1ec3c2e409f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:05.142144 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54b5a9-06ca-42ef-90d1-e1ec3c2e409f None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:05.143719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 579/1154] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:34:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:34:05.149213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92479d67-1b7c-40ef-bb46-44bbc0df6997 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:05.149717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92479d67-1b7c-40ef-bb46-44bbc0df6997 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:05.149806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92479d67-1b7c-40ef-bb46-44bbc0df6997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:05.172794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-92479d67-1b7c-40ef-bb46-44bbc0df6997 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:34:05.174036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-92479d67-1b7c-40ef-bb46-44bbc0df6997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:34:05.193872 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:05.247705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 576/1155] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:34:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:34:05.272049 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54b5a9-06ca-42ef-90d1-e1ec3c2e409f None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:05.287681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:05.379062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54b5a9-06ca-42ef-90d1-e1ec3c2e409f None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:05.416992 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2fbb9-a46a-4ca3-b8ba-eb7155e5538f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:05.417842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2fbb9-a46a-4ca3-b8ba-eb7155e5538f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:05.418126 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2fbb9-a46a-4ca3-b8ba-eb7155e5538f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:05.418401 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2fbb9-a46a-4ca3-b8ba-eb7155e5538f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:05.430317 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2fbb9-a46a-4ca3-b8ba-eb7155e5538f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:34:05.438732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-a2b2fbb9-a46a-4ca3-b8ba-eb7155e5538f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:34:05.438732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:34:05.438732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:34:05.438732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:34:05.438732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:34:05.438732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:34:05.438732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:34:05.438732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:34:05.438732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:34:05.438732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:34:05.438732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:34:05.438732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:34:05.438732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:34:05.438732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:34:05.438732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:34:05.438732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:34:05.439947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:34:05.439947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:34:05.439947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:34:05.439947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:34:05.439947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:34:05.439947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:34:05.439947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:34:05.439947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:34:05.439947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:34:05.439947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:34:05.439947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:34:05.439947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:34:05.439947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:34:05.439947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:34:05.439947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:34:05.439947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:34:05.440860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:34:05.440860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:34:05.440860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:34:05.440860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:34:05.440860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:34:05.440860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:34:05.440860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:34:05.440860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:34:05.440860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:34:05.440860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:34:05.440860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:34:05.440860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:34:05.440860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 580/1156] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:34:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:34:05.446626 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92479d67-1b7c-40ef-bb46-44bbc0df6997 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:05.557133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92479d67-1b7c-40ef-bb46-44bbc0df6997 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:05.591452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c626ebe-4a62-484e-8075-cb47c44d70de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:05.592121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c626ebe-4a62-484e-8075-cb47c44d70de None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:05.592404 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c626ebe-4a62-484e-8075-cb47c44d70de None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:05.592683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c626ebe-4a62-484e-8075-cb47c44d70de None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:05.593439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c626ebe-4a62-484e-8075-cb47c44d70de None admin] RBAC: Adding query filter params (name=admin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:34:05.593749 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c626ebe-4a62-484e-8075-cb47c44d70de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:05.595686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c626ebe-4a62-484e-8075-cb47c44d70de None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:05.605647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 577/1157] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:34:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:34:05.612500 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2fbb9-a46a-4ca3-b8ba-eb7155e5538f None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:05.686413 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:05.717242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2fbb9-a46a-4ca3-b8ba-eb7155e5538f None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:05.754618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d79192-7d82-4547-b405-616b85375993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:05.755536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d79192-7d82-4547-b405-616b85375993 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:05.755821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d79192-7d82-4547-b405-616b85375993 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:05.756100 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d79192-7d82-4547-b405-616b85375993 None admin] PATH_INFO: `/v3/users/75991f1afee34530b764244fa8e2d6e9` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:05.757389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d79192-7d82-4547-b405-616b85375993 None admin] RBAC: Authorizing `identity:get_user(user_id=75991f1afee34530b764244fa8e2d6e9)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:05.759100 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d79192-7d82-4547-b405-616b85375993 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:05.761062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 581/1158] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:34:05 2022] GET /identity/v3/users/75991f1afee34530b764244fa8e2d6e9 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:34:05.767719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c626ebe-4a62-484e-8075-cb47c44d70de None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:05.784607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:05.873245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c626ebe-4a62-484e-8075-cb47c44d70de None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:05.909763 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5206de-510d-48b9-993d-985a281f23d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:05.910416 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5206de-510d-48b9-993d-985a281f23d9 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:05.910705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5206de-510d-48b9-993d-985a281f23d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:05.910985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5206de-510d-48b9-993d-985a281f23d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:05.979718 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5206de-510d-48b9-993d-985a281f23d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:05.981074 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5206de-510d-48b9-993d-985a281f23d9 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:05.995339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:06.082403 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 578/1159] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:34:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 22 11:34:06.088361 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d79192-7d82-4547-b405-616b85375993 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:06.185474 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d79192-7d82-4547-b405-616b85375993 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:06.218403 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26eec81-e55a-43af-b931-70f265db15b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:06.219810 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26eec81-e55a-43af-b931-70f265db15b4 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:06.220097 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26eec81-e55a-43af-b931-70f265db15b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:06.220374 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26eec81-e55a-43af-b931-70f265db15b4 None admin] PATH_INFO: `/v3/projects/19f1cc7ea0fa456f9dab19a4039de405` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:06.221552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26eec81-e55a-43af-b931-70f265db15b4 None admin] RBAC: Authorizing `identity:get_project(project_id=19f1cc7ea0fa456f9dab19a4039de405)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:06.222986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26eec81-e55a-43af-b931-70f265db15b4 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:06.224711 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 582/1160] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:34:06 2022] GET /identity/v3/projects/19f1cc7ea0fa456f9dab19a4039de405 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:06.230807 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5206de-510d-48b9-993d-985a281f23d9 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:06.336197 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5206de-510d-48b9-993d-985a281f23d9 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:06.355767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:06.380452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a313e1-28a5-488b-b692-18eba78de104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:06.381100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a313e1-28a5-488b-b692-18eba78de104 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:06.381377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a313e1-28a5-488b-b692-18eba78de104 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:06.381653 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a313e1-28a5-488b-b692-18eba78de104 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:06.382477 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a313e1-28a5-488b-b692-18eba78de104 None admin] RBAC: Adding query filter params (role.id=96c584428d924fdba9671949f760c406, scope.project.id=19f1cc7ea0fa456f9dab19a4039de405, user.id=75991f1afee34530b764244fa8e2d6e9) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:34:06.382785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a313e1-28a5-488b-b692-18eba78de104 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:06.384228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a313e1-28a5-488b-b692-18eba78de104 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:06.390151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 579/1161] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:34:06 2022] GET /identity/v3/role_assignments?user.id=75991f1afee34530b764244fa8e2d6e9&scope.project.id=19f1cc7ea0fa456f9dab19a4039de405&role.id=96c584428d924fdba9671949f760c406 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:08.434629 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf479de9-a97e-432c-8823-01eabd3a4325 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:08.435265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf479de9-a97e-432c-8823-01eabd3a4325 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:08.435265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf479de9-a97e-432c-8823-01eabd3a4325 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:08.436582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 583/1162] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:34:08.442020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864eb27-90ec-4ef9-9bc0-1fa0d90db91d None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:08.442409 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864eb27-90ec-4ef9-9bc0-1fa0d90db91d None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:08.442583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864eb27-90ec-4ef9-9bc0-1fa0d90db91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:08.459599 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-1864eb27-90ec-4ef9-9bc0-1fa0d90db91d None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:34:08.460779 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-1864eb27-90ec-4ef9-9bc0-1fa0d90db91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:34:08.524757 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 580/1163] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:34:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:34:08.547629 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf479de9-a97e-432c-8823-01eabd3a4325 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:08.558387 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:08.649758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf479de9-a97e-432c-8823-01eabd3a4325 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:08.681649 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aa37d-309c-45b2-b7ac-24a7cd5daff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:08.682287 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aa37d-309c-45b2-b7ac-24a7cd5daff9 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:08.682571 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aa37d-309c-45b2-b7ac-24a7cd5daff9 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:08.682849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aa37d-309c-45b2-b7ac-24a7cd5daff9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:08.748317 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992aa37d-309c-45b2-b7ac-24a7cd5daff9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:08.749680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992aa37d-309c-45b2-b7ac-24a7cd5daff9 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:08.758926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:08.844965 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 584/1164] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:34:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 22 11:34:08.851282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864eb27-90ec-4ef9-9bc0-1fa0d90db91d None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:08.947896 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864eb27-90ec-4ef9-9bc0-1fa0d90db91d None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:08.957954 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:08.984120 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a5516-16ae-40b6-a3c3-109186d2b1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:08.984905 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a5516-16ae-40b6-a3c3-109186d2b1b3 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:08.985185 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a5516-16ae-40b6-a3c3-109186d2b1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:08.985457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a5516-16ae-40b6-a3c3-109186d2b1b3 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:08.992140 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a5516-16ae-40b6-a3c3-109186d2b1b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:08.993573 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a5516-16ae-40b6-a3c3-109186d2b1b3 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:09.007637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-a55a5516-16ae-40b6-a3c3-109186d2b1b3 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 22 11:34:09.007637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:34:09.007637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 22 11:34:09.007637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 22 11:34:09.007637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:34:09.007637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:34:09.007637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:34:09.007637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:34:09.007637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 22 11:34:09.007637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:34:09.007637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:34:09.007637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:34:09.007637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:34:09.007637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:34:09.007637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:34:09.007637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:34:09.007637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:34:09.009005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:34:09.009005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:34:09.009005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:34:09.009005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:34:09.009005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:34:09.009005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:34:09.009005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:34:09.009005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:34:09.009005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:34:09.009005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:34:09.009005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:34:09.009005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:34:09.009005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:34:09.009005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:34:09.009005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:34:09.009005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:34:09.009961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:34:09.009961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:34:09.009961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:34:09.009961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:34:09.009961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:34:09.009961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:34:09.009961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:34:09.009961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:34:09.009961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:34:09.009961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:34:09.009961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 22 11:34:09.009961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:34:09.009961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 22 11:34:09.009961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:34:09.009961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 581/1165] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jun 22 11:34:08 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:34:09.015298 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aa37d-309c-45b2-b7ac-24a7cd5daff9 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:09.114207 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aa37d-309c-45b2-b7ac-24a7cd5daff9 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:09.123180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:09.149077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f63df-d5db-4c51-916f-ef8a7c13956a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:09.149726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f63df-d5db-4c51-916f-ef8a7c13956a None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:09.150011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f63df-d5db-4c51-916f-ef8a7c13956a None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:09.150288 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f63df-d5db-4c51-916f-ef8a7c13956a None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:09.150997 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f63df-d5db-4c51-916f-ef8a7c13956a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:34:09.151303 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f63df-d5db-4c51-916f-ef8a7c13956a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:09.152695 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f63df-d5db-4c51-916f-ef8a7c13956a None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:09.159950 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 585/1166] 10.222.0.36 () {56 vars in 1115 bytes} [Wed Jun 22 11:34:09 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:11.186448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c1a5e-880d-40e9-82f8-804cb090ca25 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:11.186999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c1a5e-880d-40e9-82f8-804cb090ca25 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:11.186999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c1a5e-880d-40e9-82f8-804cb090ca25 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:11.188968 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 582/1167] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:34:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:34:11.194271 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bb7fb-27e5-4e2c-ae00-4673693eba6a None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:11.194698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bb7fb-27e5-4e2c-ae00-4673693eba6a None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:11.194799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bb7fb-27e5-4e2c-ae00-4673693eba6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:11.211199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-8e9bb7fb-27e5-4e2c-ae00-4673693eba6a None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:34:11.212418 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-8e9bb7fb-27e5-4e2c-ae00-4673693eba6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:34:11.277364 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 586/1168] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:34:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:34:11.300945 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c1a5e-880d-40e9-82f8-804cb090ca25 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:11.401595 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c1a5e-880d-40e9-82f8-804cb090ca25 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:11.434579 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f37741-dcff-4dea-acc9-2de12e8663d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:11.435235 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f37741-dcff-4dea-acc9-2de12e8663d5 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:11.435549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f37741-dcff-4dea-acc9-2de12e8663d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:11.435822 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f37741-dcff-4dea-acc9-2de12e8663d5 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:11.436552 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f37741-dcff-4dea-acc9-2de12e8663d5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:11.437910 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f37741-dcff-4dea-acc9-2de12e8663d5 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:11.452142 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 583/1169] 10.222.0.36 () {60 vars in 1138 bytes} [Wed Jun 22 11:34:11 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:34:13.591708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00037b78-3986-41a8-a34c-c75295a10a15 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:13.592226 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00037b78-3986-41a8-a34c-c75295a10a15 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:13.592343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00037b78-3986-41a8-a34c-c75295a10a15 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:13.594200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 587/1170] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:34:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:34:13.599430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe18c2-591a-4bf8-9b5d-c4c3a44263f6 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:13.599855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe18c2-591a-4bf8-9b5d-c4c3a44263f6 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:13.599992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe18c2-591a-4bf8-9b5d-c4c3a44263f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:13.616294 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-79fe18c2-591a-4bf8-9b5d-c4c3a44263f6 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:34:13.617469 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-79fe18c2-591a-4bf8-9b5d-c4c3a44263f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:34:13.639866 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:13.684232 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 584/1171] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:34:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:34:13.708743 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00037b78-3986-41a8-a34c-c75295a10a15 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:13.807921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00037b78-3986-41a8-a34c-c75295a10a15 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:13.809471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:13.843535 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8896ab-d026-40e3-b6ed-eacf28b6a8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:13.844320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8896ab-d026-40e3-b6ed-eacf28b6a8a1 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:13.844604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8896ab-d026-40e3-b6ed-eacf28b6a8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:13.844883 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8896ab-d026-40e3-b6ed-eacf28b6a8a1 None admin] PATH_INFO: `/v3/domains/a6236c97b1b1403f828559ef3b9839f8` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:13.855158 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8896ab-d026-40e3-b6ed-eacf28b6a8a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a6236c97b1b1403f828559ef3b9839f8)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:13.856637 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8896ab-d026-40e3-b6ed-eacf28b6a8a1 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:13.858273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 588/1172] 10.222.0.36 () {56 vars in 1183 bytes} [Wed Jun 22 11:34:13 2022] GET /identity/v3/domains/a6236c97b1b1403f828559ef3b9839f8 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:34:13.864318 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe18c2-591a-4bf8-9b5d-c4c3a44263f6 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:13.960290 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe18c2-591a-4bf8-9b5d-c4c3a44263f6 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:13.995214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce61fc4-b600-45ae-b8fd-a837ed75d7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:13.995896 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce61fc4-b600-45ae-b8fd-a837ed75d7de None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:13.996175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce61fc4-b600-45ae-b8fd-a837ed75d7de None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:13.996453 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce61fc4-b600-45ae-b8fd-a837ed75d7de None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:13.997158 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce61fc4-b600-45ae-b8fd-a837ed75d7de None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:13.998578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce61fc4-b600-45ae-b8fd-a837ed75d7de None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:14.030695 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 585/1173] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jun 22 11:34:13 2022] POST /identity/v3/projects => generated 361 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:34:16.104640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859df1b-4914-4142-9a43-5126c7842980 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:16.105190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859df1b-4914-4142-9a43-5126c7842980 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:16.105190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859df1b-4914-4142-9a43-5126c7842980 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:16.106633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 589/1174] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:34:16.112568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56a6fb-4dc3-4257-a866-6c88571068e9 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:16.112978 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56a6fb-4dc3-4257-a866-6c88571068e9 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:16.113123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56a6fb-4dc3-4257-a866-6c88571068e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:16.130085 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-bb56a6fb-4dc3-4257-a866-6c88571068e9 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:34:16.131277 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-bb56a6fb-4dc3-4257-a866-6c88571068e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:34:16.134061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:16.198068 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 586/1175] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:34:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:34:16.223715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859df1b-4914-4142-9a43-5126c7842980 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:16.323899 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859df1b-4914-4142-9a43-5126c7842980 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:16.356605 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718143c-52cf-4f4e-b475-d1168c45cc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:16.357456 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718143c-52cf-4f4e-b475-d1168c45cc18 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:16.357735 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718143c-52cf-4f4e-b475-d1168c45cc18 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:16.358010 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718143c-52cf-4f4e-b475-d1168c45cc18 None admin] PATH_INFO: `/v3/domains/a6236c97b1b1403f828559ef3b9839f8` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:16.359701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718143c-52cf-4f4e-b475-d1168c45cc18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a6236c97b1b1403f828559ef3b9839f8)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:16.361126 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718143c-52cf-4f4e-b475-d1168c45cc18 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:16.362741 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 590/1176] 10.222.0.36 () {56 vars in 1183 bytes} [Wed Jun 22 11:34:16 2022] GET /identity/v3/domains/a6236c97b1b1403f828559ef3b9839f8 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:16.368529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56a6fb-4dc3-4257-a866-6c88571068e9 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:16.463310 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56a6fb-4dc3-4257-a866-6c88571068e9 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:16.495799 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2face7-0a7d-415c-8666-f991c52072ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:16.496461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2face7-0a7d-415c-8666-f991c52072ef None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:16.496745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2face7-0a7d-415c-8666-f991c52072ef None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:16.497022 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2face7-0a7d-415c-8666-f991c52072ef None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:16.497717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2face7-0a7d-415c-8666-f991c52072ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:16.499131 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2face7-0a7d-415c-8666-f991c52072ef None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:16.508864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:16.523997 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 587/1177] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jun 22 11:34:16 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:34:18.601082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f989db7-1bed-4f79-ba97-2eb5b6b7e509 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:18.601608 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f989db7-1bed-4f79-ba97-2eb5b6b7e509 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:18.601722 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f989db7-1bed-4f79-ba97-2eb5b6b7e509 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:18.602974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 591/1178] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:34:18.608277 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12b5a4-59a3-4392-bd36-c8d5ddfc6d7c None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:18.608657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12b5a4-59a3-4392-bd36-c8d5ddfc6d7c None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:18.608964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12b5a4-59a3-4392-bd36-c8d5ddfc6d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:18.626054 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-1c12b5a4-59a3-4392-bd36-c8d5ddfc6d7c None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:34:18.627246 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-1c12b5a4-59a3-4392-bd36-c8d5ddfc6d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:34:18.692716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 588/1179] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:34:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:34:18.716737 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f989db7-1bed-4f79-ba97-2eb5b6b7e509 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:18.747076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:18.818116 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f989db7-1bed-4f79-ba97-2eb5b6b7e509 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:18.851390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b37dfe-56fb-4698-af3a-a0506857365f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:18.852109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b37dfe-56fb-4698-af3a-a0506857365f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:18.852389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b37dfe-56fb-4698-af3a-a0506857365f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:18.852658 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b37dfe-56fb-4698-af3a-a0506857365f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:18.862874 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b37dfe-56fb-4698-af3a-a0506857365f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:34:18.870273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-a1b37dfe-56fb-4698-af3a-a0506857365f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:34:18.870273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:34:18.870273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:34:18.870273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:34:18.870273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:34:18.870273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:34:18.870273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:34:18.870273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:34:18.870273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:34:18.870273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:34:18.870273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:34:18.870273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:34:18.870273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:34:18.870273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:34:18.870273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:34:18.870273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:34:18.871433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:34:18.871433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:34:18.871433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:34:18.871433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:34:18.871433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:34:18.871433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:34:18.871433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:34:18.871433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:34:18.871433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:34:18.871433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:34:18.871433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:34:18.871433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:34:18.871433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:34:18.871433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:34:18.871433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:34:18.871433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:34:18.872485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:34:18.872485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:34:18.872485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:34:18.872485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:34:18.872485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:34:18.872485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:34:18.872485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:34:18.872485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:34:18.872485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:34:18.872485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:34:18.872485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:34:18.872485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:34:18.872485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 592/1180] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:34:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:34:18.879414 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12b5a4-59a3-4392-bd36-c8d5ddfc6d7c None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:18.976300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12b5a4-59a3-4392-bd36-c8d5ddfc6d7c None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:19.000983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:19.009713 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b0fe0-f886-4e2f-9945-3216b4c64e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:19.010372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b0fe0-f886-4e2f-9945-3216b4c64e13 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:19.010667 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b0fe0-f886-4e2f-9945-3216b4c64e13 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:19.010922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b0fe0-f886-4e2f-9945-3216b4c64e13 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:19.011689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b0fe0-f886-4e2f-9945-3216b4c64e13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:34:19.011990 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b0fe0-f886-4e2f-9945-3216b4c64e13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:19.013332 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b0fe0-f886-4e2f-9945-3216b4c64e13 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:19.021386 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 589/1181] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:34:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:19.027298 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b37dfe-56fb-4698-af3a-a0506857365f None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:19.116043 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:19.125470 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b37dfe-56fb-4698-af3a-a0506857365f None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:19.158308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d7225-44af-484d-aeab-20a9c713e353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:19.159162 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d7225-44af-484d-aeab-20a9c713e353 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:19.159465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d7225-44af-484d-aeab-20a9c713e353 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:19.159741 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d7225-44af-484d-aeab-20a9c713e353 None admin] PATH_INFO: `/v3/users/aaf93aecfb1c44a1be61bd62ffa6c2f9` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:19.172659 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d7225-44af-484d-aeab-20a9c713e353 None admin] RBAC: Authorizing `identity:get_user(user_id=aaf93aecfb1c44a1be61bd62ffa6c2f9)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:19.174087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d7225-44af-484d-aeab-20a9c713e353 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:19.175740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 593/1182] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:34:19 2022] GET /identity/v3/users/aaf93aecfb1c44a1be61bd62ffa6c2f9 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:19.181544 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b0fe0-f886-4e2f-9945-3216b4c64e13 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:19.276909 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b0fe0-f886-4e2f-9945-3216b4c64e13 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:19.309963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58f37a-221b-45c4-8850-5b26dbd37657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:19.310597 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58f37a-221b-45c4-8850-5b26dbd37657 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:19.310886 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58f37a-221b-45c4-8850-5b26dbd37657 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:19.311157 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58f37a-221b-45c4-8850-5b26dbd37657 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:19.364651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:19.378186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58f37a-221b-45c4-8850-5b26dbd37657 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:19.380892 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58f37a-221b-45c4-8850-5b26dbd37657 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:19.478997 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 590/1183] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:34:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 22 11:34:19.484886 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d7225-44af-484d-aeab-20a9c713e353 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:19.581977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d7225-44af-484d-aeab-20a9c713e353 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:19.614696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf89f55c-a760-4cfc-b0ec-ae760ce27878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:19.615570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89f55c-a760-4cfc-b0ec-ae760ce27878 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:19.615848 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89f55c-a760-4cfc-b0ec-ae760ce27878 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:19.616120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf89f55c-a760-4cfc-b0ec-ae760ce27878 None admin] PATH_INFO: `/v3/projects/28e9cdc9c1624ca58759fc422f647cce` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:19.617293 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89f55c-a760-4cfc-b0ec-ae760ce27878 None admin] RBAC: Authorizing `identity:get_project(project_id=28e9cdc9c1624ca58759fc422f647cce)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:19.618722 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf89f55c-a760-4cfc-b0ec-ae760ce27878 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:19.620421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 594/1184] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:34:19 2022] GET /identity/v3/projects/28e9cdc9c1624ca58759fc422f647cce => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:19.627124 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58f37a-221b-45c4-8850-5b26dbd37657 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:19.710927 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:19.724156 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58f37a-221b-45c4-8850-5b26dbd37657 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:19.756901 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119dd7c-6465-4061-ace8-c519cdf49eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:19.757547 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119dd7c-6465-4061-ace8-c519cdf49eef None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:19.757831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119dd7c-6465-4061-ace8-c519cdf49eef None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:19.758109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119dd7c-6465-4061-ace8-c519cdf49eef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:19.758930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119dd7c-6465-4061-ace8-c519cdf49eef None admin] RBAC: Adding query filter params (role.id=96c584428d924fdba9671949f760c406, scope.project.id=28e9cdc9c1624ca58759fc422f647cce, user.id=aaf93aecfb1c44a1be61bd62ffa6c2f9) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:34:19.759234 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119dd7c-6465-4061-ace8-c519cdf49eef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:19.760677 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119dd7c-6465-4061-ace8-c519cdf49eef None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:19.766722 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 591/1185] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:34:19 2022] GET /identity/v3/role_assignments?user.id=aaf93aecfb1c44a1be61bd62ffa6c2f9&scope.project.id=28e9cdc9c1624ca58759fc422f647cce&role.id=96c584428d924fdba9671949f760c406 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:21.803096 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a342e-b65f-44ac-a594-e9c7d187512c None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:21.803655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a342e-b65f-44ac-a594-e9c7d187512c None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:21.803655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a342e-b65f-44ac-a594-e9c7d187512c None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:21.805028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 595/1186] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:34:21.810114 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101eeef-b72e-464c-901e-45f282a32afb None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:21.810553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101eeef-b72e-464c-901e-45f282a32afb None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:21.810667 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101eeef-b72e-464c-901e-45f282a32afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:21.827663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-2101eeef-b72e-464c-901e-45f282a32afb None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:34:21.828845 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2101eeef-b72e-464c-901e-45f282a32afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:34:21.892865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 592/1187] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:34:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:34:21.915223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a342e-b65f-44ac-a594-e9c7d187512c None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:21.934205 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:22.017118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a342e-b65f-44ac-a594-e9c7d187512c None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:22.050194 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641ca83-5900-4959-a483-1a6fdf70889f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:22.050974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641ca83-5900-4959-a483-1a6fdf70889f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:22.051262 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641ca83-5900-4959-a483-1a6fdf70889f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:22.051567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641ca83-5900-4959-a483-1a6fdf70889f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:22.061650 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641ca83-5900-4959-a483-1a6fdf70889f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:34:22.069888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-1641ca83-5900-4959-a483-1a6fdf70889f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:34:22.069888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:34:22.069888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:34:22.069888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:34:22.069888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:34:22.069888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:34:22.069888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:34:22.069888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:34:22.069888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:34:22.069888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:34:22.069888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:34:22.069888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:34:22.069888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:34:22.069888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:34:22.069888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:34:22.069888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:34:22.071032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:34:22.071032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:34:22.071032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:34:22.071032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:34:22.071032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:34:22.071032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:34:22.071032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:34:22.071032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:34:22.071032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:34:22.071032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:34:22.071032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:34:22.071032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:34:22.071032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:34:22.071032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:34:22.071032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:34:22.071032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:34:22.071955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:34:22.071955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:34:22.071955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:34:22.071955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:34:22.071955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:34:22.071955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:34:22.071955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:34:22.071955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:34:22.071955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:34:22.071955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:34:22.071955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:34:22.071955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:34:22.071955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 596/1188] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:34:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:34:22.077282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101eeef-b72e-464c-901e-45f282a32afb None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:22.158867 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:22.176575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101eeef-b72e-464c-901e-45f282a32afb None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:22.209465 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940099d-37c0-4db4-934e-11e514486914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:22.210119 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940099d-37c0-4db4-934e-11e514486914 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:22.210403 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940099d-37c0-4db4-934e-11e514486914 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:22.210683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940099d-37c0-4db4-934e-11e514486914 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:22.211448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940099d-37c0-4db4-934e-11e514486914 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:34:22.211756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940099d-37c0-4db4-934e-11e514486914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:22.213112 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940099d-37c0-4db4-934e-11e514486914 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:22.221064 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 593/1189] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:34:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:22.226825 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641ca83-5900-4959-a483-1a6fdf70889f None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:22.301828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:22.324551 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641ca83-5900-4959-a483-1a6fdf70889f None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:22.357198 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297fdbe-9ea5-4850-a303-1affc46c97d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:22.358058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297fdbe-9ea5-4850-a303-1affc46c97d0 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:22.358335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297fdbe-9ea5-4850-a303-1affc46c97d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:22.358605 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297fdbe-9ea5-4850-a303-1affc46c97d0 None admin] PATH_INFO: `/v3/users/aaf93aecfb1c44a1be61bd62ffa6c2f9` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:22.360323 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1297fdbe-9ea5-4850-a303-1affc46c97d0 None admin] RBAC: Authorizing `identity:get_user(user_id=aaf93aecfb1c44a1be61bd62ffa6c2f9)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:22.361753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1297fdbe-9ea5-4850-a303-1affc46c97d0 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:22.363401 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 597/1190] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:34:22 2022] GET /identity/v3/users/aaf93aecfb1c44a1be61bd62ffa6c2f9 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:22.369133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940099d-37c0-4db4-934e-11e514486914 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:22.465512 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940099d-37c0-4db4-934e-11e514486914 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:22.498298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83344a99-dca6-4488-b8fc-ab693566a2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:22.499144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83344a99-dca6-4488-b8fc-ab693566a2a5 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:22.500018 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83344a99-dca6-4488-b8fc-ab693566a2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:22.500298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83344a99-dca6-4488-b8fc-ab693566a2a5 None admin] PATH_INFO: `/v3/projects/28e9cdc9c1624ca58759fc422f647cce` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:22.501475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83344a99-dca6-4488-b8fc-ab693566a2a5 None admin] RBAC: Authorizing `identity:get_project(project_id=28e9cdc9c1624ca58759fc422f647cce)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:22.502893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83344a99-dca6-4488-b8fc-ab693566a2a5 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:22.504621 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 594/1191] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:34:22 2022] GET /identity/v3/projects/28e9cdc9c1624ca58759fc422f647cce => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:22.510618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297fdbe-9ea5-4850-a303-1affc46c97d0 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:22.608652 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297fdbe-9ea5-4850-a303-1affc46c97d0 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:22.642031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365f5c4-cd1b-42f4-bc61-a054ad23b6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:22.642909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365f5c4-cd1b-42f4-bc61-a054ad23b6de None admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:22.643190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365f5c4-cd1b-42f4-bc61-a054ad23b6de None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:22.643495 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365f5c4-cd1b-42f4-bc61-a054ad23b6de None admin] PATH_INFO: `/v3/projects/28e9cdc9c1624ca58759fc422f647cce/users/aaf93aecfb1c44a1be61bd62ffa6c2f9/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:22.645451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365f5c4-cd1b-42f4-bc61-a054ad23b6de None admin] RBAC: Authorizing `identity:create_grant(project_id=28e9cdc9c1624ca58759fc422f647cce, user_id=aaf93aecfb1c44a1be61bd62ffa6c2f9, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:22.646861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365f5c4-cd1b-42f4-bc61-a054ad23b6de None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:22.659483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 598/1192] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:34:22 2022] PUT /identity/v3/projects/28e9cdc9c1624ca58759fc422f647cce/users/aaf93aecfb1c44a1be61bd62ffa6c2f9/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:34:24.701963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d58cc-59ea-48be-b3f7-0746d0e2ae71 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:24.702651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d58cc-59ea-48be-b3f7-0746d0e2ae71 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:24.702651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d58cc-59ea-48be-b3f7-0746d0e2ae71 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:24.703926 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 595/1193] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:34:24.709201 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96163e69-fd7a-4751-be3c-28c708b5b0ac None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:24.709599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96163e69-fd7a-4751-be3c-28c708b5b0ac None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:24.709765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96163e69-fd7a-4751-be3c-28c708b5b0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:24.715852 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:24.728325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-96163e69-fd7a-4751-be3c-28c708b5b0ac None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:34:24.729518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-96163e69-fd7a-4751-be3c-28c708b5b0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:34:24.794267 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 599/1194] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:34:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:34:24.818758 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d58cc-59ea-48be-b3f7-0746d0e2ae71 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:24.834134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:24.918317 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d58cc-59ea-48be-b3f7-0746d0e2ae71 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:24.953461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60eb422-055b-4807-bd2a-7e2bba34a0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:24.954186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60eb422-055b-4807-bd2a-7e2bba34a0f0 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:24.954469 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60eb422-055b-4807-bd2a-7e2bba34a0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:24.954746 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60eb422-055b-4807-bd2a-7e2bba34a0f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:24.965097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60eb422-055b-4807-bd2a-7e2bba34a0f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:34:24.973218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-d60eb422-055b-4807-bd2a-7e2bba34a0f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:34:24.973218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:34:24.973218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:34:24.973218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:34:24.973218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:34:24.973218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:34:24.973218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:34:24.973218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:34:24.973218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:34:24.973218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:34:24.973218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:34:24.973218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:34:24.973218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:34:24.973218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:34:24.973218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:34:24.973218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:34:24.974188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:34:24.974188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:34:24.974188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:34:24.974188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:34:24.974188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:34:24.974188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:34:24.974188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:34:24.974188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:34:24.974188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:34:24.974188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:34:24.974188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:34:24.974188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:34:24.974188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:34:24.974188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:34:24.974188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:34:24.974188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:34:24.975165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:34:24.975165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:34:24.975165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:34:24.975165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:34:24.975165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:34:24.975165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:34:24.975165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:34:24.975165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:34:24.975165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:34:24.975165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:34:24.975165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:34:24.975165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:34:24.975165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 596/1195] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:34:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:34:24.980429 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96163e69-fd7a-4751-be3c-28c708b5b0ac None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:25.076096 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96163e69-fd7a-4751-be3c-28c708b5b0ac None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:25.108735 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5d34e-73b8-48d4-a8f5-7e779efe8df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:25.109400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5d34e-73b8-48d4-a8f5-7e779efe8df1 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:25.109681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5d34e-73b8-48d4-a8f5-7e779efe8df1 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:25.109960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5d34e-73b8-48d4-a8f5-7e779efe8df1 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:25.110701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5d34e-73b8-48d4-a8f5-7e779efe8df1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:34:25.111006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5d34e-73b8-48d4-a8f5-7e779efe8df1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:25.112413 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5d34e-73b8-48d4-a8f5-7e779efe8df1 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:25.113788 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:25.121131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 600/1196] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:34:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:25.127669 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60eb422-055b-4807-bd2a-7e2bba34a0f0 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:25.196057 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:25.225015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60eb422-055b-4807-bd2a-7e2bba34a0f0 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:25.257213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f39d8-61c3-440f-94b8-de54c8d96941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:25.258067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f39d8-61c3-440f-94b8-de54c8d96941 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:25.258411 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f39d8-61c3-440f-94b8-de54c8d96941 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:25.258706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f39d8-61c3-440f-94b8-de54c8d96941 None admin] PATH_INFO: `/v3/users/aaf93aecfb1c44a1be61bd62ffa6c2f9` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:25.259899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f39d8-61c3-440f-94b8-de54c8d96941 None admin] RBAC: Authorizing `identity:get_user(user_id=aaf93aecfb1c44a1be61bd62ffa6c2f9)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:25.261314 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f39d8-61c3-440f-94b8-de54c8d96941 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:25.262944 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 597/1197] 10.222.0.36 () {56 vars in 1177 bytes} [Wed Jun 22 11:34:25 2022] GET /identity/v3/users/aaf93aecfb1c44a1be61bd62ffa6c2f9 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:25.269552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5d34e-73b8-48d4-a8f5-7e779efe8df1 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:25.373699 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5d34e-73b8-48d4-a8f5-7e779efe8df1 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:25.405972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0760d3-af2b-4bd4-a8ed-e30ebf206209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:25.406609 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0760d3-af2b-4bd4-a8ed-e30ebf206209 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:25.406888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0760d3-af2b-4bd4-a8ed-e30ebf206209 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:25.407160 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0760d3-af2b-4bd4-a8ed-e30ebf206209 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:25.465836 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:25.472924 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0760d3-af2b-4bd4-a8ed-e30ebf206209 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:25.474359 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0760d3-af2b-4bd4-a8ed-e30ebf206209 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:25.572963 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 601/1198] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:34:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 22 11:34:25.579414 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f39d8-61c3-440f-94b8-de54c8d96941 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:25.683203 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f39d8-61c3-440f-94b8-de54c8d96941 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:25.715407 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d958c-66dc-4011-91b5-ae803e2dcb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:25.716254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d958c-66dc-4011-91b5-ae803e2dcb36 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:25.716532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d958c-66dc-4011-91b5-ae803e2dcb36 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:25.716805 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d958c-66dc-4011-91b5-ae803e2dcb36 None admin] PATH_INFO: `/v3/projects/28e9cdc9c1624ca58759fc422f647cce` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:25.718000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d958c-66dc-4011-91b5-ae803e2dcb36 None admin] RBAC: Authorizing `identity:get_project(project_id=28e9cdc9c1624ca58759fc422f647cce)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:25.720045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d958c-66dc-4011-91b5-ae803e2dcb36 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:25.721729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 598/1199] 10.222.0.36 () {56 vars in 1186 bytes} [Wed Jun 22 11:34:25 2022] GET /identity/v3/projects/28e9cdc9c1624ca58759fc422f647cce => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:25.727872 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0760d3-af2b-4bd4-a8ed-e30ebf206209 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:34:25.822318 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:25.829267 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0760d3-af2b-4bd4-a8ed-e30ebf206209 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:34:25.861808 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6936ee-e36d-47c0-9462-b1173203a0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:34:25.862455 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6936ee-e36d-47c0-9462-b1173203a0f4 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:34:25.862738 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6936ee-e36d-47c0-9462-b1173203a0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:34:25.863010 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6936ee-e36d-47c0-9462-b1173203a0f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:34:25.863857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6936ee-e36d-47c0-9462-b1173203a0f4 None admin] RBAC: Adding query filter params (role.id=96c584428d924fdba9671949f760c406, scope.project.id=28e9cdc9c1624ca58759fc422f647cce, user.id=aaf93aecfb1c44a1be61bd62ffa6c2f9) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:34:25.864166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6936ee-e36d-47c0-9462-b1173203a0f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:34:25.865594 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6936ee-e36d-47c0-9462-b1173203a0f4 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:34:25.871380 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 602/1200] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:34:25 2022] GET /identity/v3/role_assignments?user.id=aaf93aecfb1c44a1be61bd62ffa6c2f9&scope.project.id=28e9cdc9c1624ca58759fc422f647cce&role.id=96c584428d924fdba9671949f760c406 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:34:35.711304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=96 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:34:35.868660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=20 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:23.358347 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00ea80-dad7-435d-88c2-e64248acf312 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:23.358932 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00ea80-dad7-435d-88c2-e64248acf312 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:23.358932 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00ea80-dad7-435d-88c2-e64248acf312 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:23.360662 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 599/1201] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:35:23.366713 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcd036-0d3a-4c51-ba57-f32f1a64a7f1 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:23.367246 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcd036-0d3a-4c51-ba57-f32f1a64a7f1 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:23.367368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcd036-0d3a-4c51-ba57-f32f1a64a7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:23.392258 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d8fcd036-0d3a-4c51-ba57-f32f1a64a7f1 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:35:23.393558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d8fcd036-0d3a-4c51-ba57-f32f1a64a7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:35:23.480245 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 603/1202] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:35:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:35:23.505944 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00ea80-dad7-435d-88c2-e64248acf312 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:23.679258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00ea80-dad7-435d-88c2-e64248acf312 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:23.720510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cff2e6-6b54-4bca-9985-bc641dfdb051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:23.721548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cff2e6-6b54-4bca-9985-bc641dfdb051 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:23.721842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cff2e6-6b54-4bca-9985-bc641dfdb051 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:23.722279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cff2e6-6b54-4bca-9985-bc641dfdb051 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:23.745300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cff2e6-6b54-4bca-9985-bc641dfdb051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:23.747551 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cff2e6-6b54-4bca-9985-bc641dfdb051 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:23.767601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-91cff2e6-6b54-4bca-9985-bc641dfdb051 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:23.767601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:23.767601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:23.767601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:23.767601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:23.767601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:23.767601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:23.767601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:23.767601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:23.767601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:23.767601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:23.767601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:23.767601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:35:23.767601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:35:23.767601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:35:23.768690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:35:23.768690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:23.768690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:23.768690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:23.768690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:23.768690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:23.768690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:23.768690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:23.768690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:23.768690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:23.768690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:23.768690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:23.768690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:23.768690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:23.768690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:23.768690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:23.768690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:23.769706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:35:23.769706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:35:23.769706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:23.769706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:35:23.774422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 600/1203] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:23.776217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcd036-0d3a-4c51-ba57-f32f1a64a7f1 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:23.913573 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcd036-0d3a-4c51-ba57-f32f1a64a7f1 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:23.957220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:23.965354 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca285de-c982-4101-abe1-f93028095d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:23.966537 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca285de-c982-4101-abe1-f93028095d6d None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:23.966888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca285de-c982-4101-abe1-f93028095d6d None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:23.967720 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca285de-c982-4101-abe1-f93028095d6d None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:23.968041 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca285de-c982-4101-abe1-f93028095d6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:23.968332 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca285de-c982-4101-abe1-f93028095d6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:23.970093 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca285de-c982-4101-abe1-f93028095d6d None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:23.989056 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 604/1204] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:35:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:23.992189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cff2e6-6b54-4bca-9985-bc641dfdb051 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:24.065641 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:24.123827 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cff2e6-6b54-4bca-9985-bc641dfdb051 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:24.170550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d72ab5-23e8-4801-8c41-0f594ec15412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:24.171250 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d72ab5-23e8-4801-8c41-0f594ec15412 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:24.171563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d72ab5-23e8-4801-8c41-0f594ec15412 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:24.171841 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d72ab5-23e8-4801-8c41-0f594ec15412 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:24.172546 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d72ab5-23e8-4801-8c41-0f594ec15412 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:24.174286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d72ab5-23e8-4801-8c41-0f594ec15412 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:24.201193 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 601/1205] 10.222.0.36 () {60 vars in 1132 bytes} [Wed Jun 22 11:35:23 2022] POST /identity/v3/users => generated 251 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:35:26.464462 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d346db-ebd9-438e-a536-2885c6b10ce3 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:26.465027 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d346db-ebd9-438e-a536-2885c6b10ce3 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:26.465132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d346db-ebd9-438e-a536-2885c6b10ce3 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:26.467206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 605/1206] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:35:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:35:26.473210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfcc27-418e-4eec-ab4b-d8178e35fe16 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:26.473669 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfcc27-418e-4eec-ab4b-d8178e35fe16 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:26.473773 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbfcc27-418e-4eec-ab4b-d8178e35fe16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:26.492321 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-3bbfcc27-418e-4eec-ab4b-d8178e35fe16 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:35:26.493545 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-3bbfcc27-418e-4eec-ab4b-d8178e35fe16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:35:26.595658 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 602/1207] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:35:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:35:26.623625 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d346db-ebd9-438e-a536-2885c6b10ce3 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:26.744622 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d346db-ebd9-438e-a536-2885c6b10ce3 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:26.781044 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76481156-86f0-427f-831b-7ad56e71ff70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:26.781868 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76481156-86f0-427f-831b-7ad56e71ff70 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:26.782149 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76481156-86f0-427f-831b-7ad56e71ff70 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:26.782432 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76481156-86f0-427f-831b-7ad56e71ff70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:26.794856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76481156-86f0-427f-831b-7ad56e71ff70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:35:26.805821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-76481156-86f0-427f-831b-7ad56e71ff70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:35:26.805821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:26.805821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:26.805821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:26.805821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:26.805821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:26.805821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:26.805821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:26.805821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:26.805821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:26.805821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:26.805821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:26.805821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:35:26.805821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:35:26.805821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:35:26.805821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:35:26.807552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:35:26.807552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:35:26.807552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:35:26.807552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:35:26.807552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:26.807552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:26.807552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:26.807552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:26.807552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:26.807552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:26.807552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:26.807552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:26.807552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:26.807552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:26.807552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:26.807552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:26.808767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:26.808767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:26.808767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:26.808767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:26.808767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:35:26.808767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:35:26.808767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:35:26.808767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:35:26.808767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:35:26.808767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:35:26.808767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:35:26.808767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:26.808767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 606/1208] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:35:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:35:26.814176 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbfcc27-418e-4eec-ab4b-d8178e35fe16 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:26.867071 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:26.951875 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbfcc27-418e-4eec-ab4b-d8178e35fe16 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:27.006725 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef76ac-52ff-4e71-b4cf-fef133c37fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:27.007748 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef76ac-52ff-4e71-b4cf-fef133c37fda None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:27.008046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef76ac-52ff-4e71-b4cf-fef133c37fda None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:27.008308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef76ac-52ff-4e71-b4cf-fef133c37fda None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:27.009073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef76ac-52ff-4e71-b4cf-fef133c37fda None admin] RBAC: Adding query filter params (name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:27.009378 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef76ac-52ff-4e71-b4cf-fef133c37fda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:27.010754 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef76ac-52ff-4e71-b4cf-fef133c37fda None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:27.020160 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 603/1209] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:35:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:27.025997 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76481156-86f0-427f-831b-7ad56e71ff70 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:27.086075 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:27.155892 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76481156-86f0-427f-831b-7ad56e71ff70 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:27.214466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37971b2-8d38-47e0-ae6e-8647f2446b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:27.216356 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37971b2-8d38-47e0-ae6e-8647f2446b9f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:27.216835 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37971b2-8d38-47e0-ae6e-8647f2446b9f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:27.217317 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37971b2-8d38-47e0-ae6e-8647f2446b9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:27.234701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37971b2-8d38-47e0-ae6e-8647f2446b9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:27.237139 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37971b2-8d38-47e0-ae6e-8647f2446b9f None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:27.263723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-e37971b2-8d38-47e0-ae6e-8647f2446b9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:27.263723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:27.263723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:27.263723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:27.263723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:27.263723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:27.263723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:27.263723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:27.263723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:27.263723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:27.263723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:27.263723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:27.263723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:35:27.263723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:35:27.263723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:35:27.265959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:35:27.265959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:27.265959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:27.265959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:27.265959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:27.265959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:27.265959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:27.265959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:27.265959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:27.265959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:27.265959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:27.265959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:27.265959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:27.265959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:27.265959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:27.265959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:27.265959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:27.266900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:35:27.266900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:35:27.266900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:27.266900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:27.266900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 607/1210] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:27.272255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef76ac-52ff-4e71-b4cf-fef133c37fda None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:27.398142 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:27.402149 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef76ac-52ff-4e71-b4cf-fef133c37fda None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:27.444130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1099990-5b4f-4e0e-852e-206a0c6ed6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:27.445040 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1099990-5b4f-4e0e-852e-206a0c6ed6bb None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:27.445412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1099990-5b4f-4e0e-852e-206a0c6ed6bb None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:27.445770 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1099990-5b4f-4e0e-852e-206a0c6ed6bb None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:27.446607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1099990-5b4f-4e0e-852e-206a0c6ed6bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:27.446987 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1099990-5b4f-4e0e-852e-206a0c6ed6bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:27.448588 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1099990-5b4f-4e0e-852e-206a0c6ed6bb None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:27.463049 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 604/1211] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:27.470089 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37971b2-8d38-47e0-ae6e-8647f2446b9f None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:27.603534 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37971b2-8d38-47e0-ae6e-8647f2446b9f None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:27.605338 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:27.653877 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66732f88-5706-4ed5-a346-4ae9b88d94e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:27.655311 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66732f88-5706-4ed5-a346-4ae9b88d94e4 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:27.655687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66732f88-5706-4ed5-a346-4ae9b88d94e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:27.656108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66732f88-5706-4ed5-a346-4ae9b88d94e4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:27.665262 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66732f88-5706-4ed5-a346-4ae9b88d94e4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:27.666909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66732f88-5706-4ed5-a346-4ae9b88d94e4 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:27.685141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-66732f88-5706-4ed5-a346-4ae9b88d94e4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 22 11:35:27.685141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:27.685141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:27.685141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:27.685141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:27.685141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:27.685141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:27.685141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:27.685141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:27.685141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:27.685141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:27.685141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:27.685141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:35:27.685141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:35:27.685141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:35:27.685141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:35:27.686508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:27.686508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:27.686508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:35:27.686508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:35:27.686508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:35:27.686508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:35:27.686508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:27.686508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:27.686508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:27.686508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:27.686508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:27.686508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:27.686508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:27.686508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:27.686508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:27.686508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:27.686508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:27.686508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:27.687623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:27.687623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:27.687623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:35:27.687623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:35:27.687623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:35:27.687623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:35:27.687623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:35:27.687623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:35:27.687623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 22 11:35:27.687623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:27.687623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 608/1212] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:27.691753 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1099990-5b4f-4e0e-852e-206a0c6ed6bb None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:27.817242 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1099990-5b4f-4e0e-852e-206a0c6ed6bb None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:27.858601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab9422d-5066-44b5-9d8d-2ec3f8dc6e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:27.859233 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9422d-5066-44b5-9d8d-2ec3f8dc6e4a None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:27.859531 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9422d-5066-44b5-9d8d-2ec3f8dc6e4a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:27.859896 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9422d-5066-44b5-9d8d-2ec3f8dc6e4a None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:27.860938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab9422d-5066-44b5-9d8d-2ec3f8dc6e4a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:27.861353 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab9422d-5066-44b5-9d8d-2ec3f8dc6e4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:27.862964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab9422d-5066-44b5-9d8d-2ec3f8dc6e4a None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:27.883448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 605/1213] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jun 22 11:35:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:27.891089 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66732f88-5706-4ed5-a346-4ae9b88d94e4 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:28.006600 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66732f88-5706-4ed5-a346-4ae9b88d94e4 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:28.046267 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d4b2d-2888-4db3-9c16-0a86c4acea2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:28.046977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d4b2d-2888-4db3-9c16-0a86c4acea2f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:28.047265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d4b2d-2888-4db3-9c16-0a86c4acea2f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:28.047587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d4b2d-2888-4db3-9c16-0a86c4acea2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:28.090076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:28.133109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d4b2d-2888-4db3-9c16-0a86c4acea2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:28.134616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d4b2d-2888-4db3-9c16-0a86c4acea2f None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:28.245603 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 609/1214] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:35:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 22 11:35:28.255843 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab9422d-5066-44b5-9d8d-2ec3f8dc6e4a None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:28.276478 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:28.369475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab9422d-5066-44b5-9d8d-2ec3f8dc6e4a None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:28.411175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ddd0af-a9b8-4b37-8257-fba8b010761d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:28.412184 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddd0af-a9b8-4b37-8257-fba8b010761d None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:28.412548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddd0af-a9b8-4b37-8257-fba8b010761d None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:28.412941 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ddd0af-a9b8-4b37-8257-fba8b010761d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:28.424382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ddd0af-a9b8-4b37-8257-fba8b010761d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:28.425918 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ddd0af-a9b8-4b37-8257-fba8b010761d None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:28.445690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-99ddd0af-a9b8-4b37-8257-fba8b010761d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:28.445690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:28.445690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:28.445690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:28.445690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:28.445690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:28.445690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:28.445690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:28.445690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:28.445690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:28.445690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:28.445690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:28.445690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:35:28.445690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:35:28.445690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:35:28.446838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:35:28.446838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:28.446838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:28.446838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:28.446838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:28.446838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:28.446838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:28.446838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:28.446838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:28.446838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:28.446838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:28.446838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:28.446838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:28.446838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:28.446838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:28.446838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:28.446838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:28.447982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:35:28.447982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:35:28.447982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:28.447982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:35:28.448204 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 606/1215] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:28.453483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d4b2d-2888-4db3-9c16-0a86c4acea2f None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:28.522285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:28.567330 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d4b2d-2888-4db3-9c16-0a86c4acea2f None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:28.610379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadd456-ea71-4f77-afc7-dfdee1ae658a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:28.611068 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadd456-ea71-4f77-afc7-dfdee1ae658a None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:28.611482 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadd456-ea71-4f77-afc7-dfdee1ae658a None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:28.611769 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cadd456-ea71-4f77-afc7-dfdee1ae658a None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:28.612730 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cadd456-ea71-4f77-afc7-dfdee1ae658a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:28.613181 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cadd456-ea71-4f77-afc7-dfdee1ae658a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:28.614940 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cadd456-ea71-4f77-afc7-dfdee1ae658a None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:28.631782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 610/1216] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:28.638382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ddd0af-a9b8-4b37-8257-fba8b010761d None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:28.721935 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:28.755984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ddd0af-a9b8-4b37-8257-fba8b010761d None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:28.797965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e49fa-62f8-41e9-a43e-39a1dcd7ed3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:28.798866 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e49fa-62f8-41e9-a43e-39a1dcd7ed3a None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:28.799162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e49fa-62f8-41e9-a43e-39a1dcd7ed3a None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:28.799473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e49fa-62f8-41e9-a43e-39a1dcd7ed3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:28.810174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e49fa-62f8-41e9-a43e-39a1dcd7ed3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:28.810174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e49fa-62f8-41e9-a43e-39a1dcd7ed3a None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:28.825566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-bc5e49fa-62f8-41e9-a43e-39a1dcd7ed3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:35:28.825566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:28.825566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:28.825566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:28.825566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:28.825566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:28.825566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:28.825566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:28.825566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:28.825566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:28.825566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:28.825566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:28.825566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:35:28.825566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:35:28.825566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:35:28.826624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:35:28.826624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:28.826624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:28.826624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:28.826624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:28.826624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:28.826624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:28.826624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:28.826624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:28.826624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:28.826624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:28.826624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:28.826624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:28.826624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:28.826624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:28.826624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:28.826624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:28.827575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:35:28.827575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:35:28.827575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:35:28.827575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:35:28.827575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:35:28.827575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:35:28.827575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:35:28.827575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:35:28.827575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 607/1217] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:35:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:28.832690 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadd456-ea71-4f77-afc7-dfdee1ae658a None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:28.961025 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cadd456-ea71-4f77-afc7-dfdee1ae658a None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:28.987170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:29.003653 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d417f5-3a96-4f8e-a916-9226d25d7502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:29.004327 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d417f5-3a96-4f8e-a916-9226d25d7502 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:29.004650 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d417f5-3a96-4f8e-a916-9226d25d7502 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:29.004868 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d417f5-3a96-4f8e-a916-9226d25d7502 None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:29.005742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d417f5-3a96-4f8e-a916-9226d25d7502 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:29.006048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d417f5-3a96-4f8e-a916-9226d25d7502 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:29.007587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d417f5-3a96-4f8e-a916-9226d25d7502 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:29.024868 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 611/1218] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:35:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:29.031859 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e49fa-62f8-41e9-a43e-39a1dcd7ed3a None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:29.151677 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e49fa-62f8-41e9-a43e-39a1dcd7ed3a None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:29.185071 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:29.191274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949fb3ad-b96a-45f2-897a-4b7ba83e8d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:29.191997 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fb3ad-b96a-45f2-897a-4b7ba83e8d96 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:29.192277 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fb3ad-b96a-45f2-897a-4b7ba83e8d96 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:29.192550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949fb3ad-b96a-45f2-897a-4b7ba83e8d96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:29.193369 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fb3ad-b96a-45f2-897a-4b7ba83e8d96 None admin] RBAC: Adding query filter params (role.id=64c2d69f15ca458c8267c1acfd42d9fc, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=93630ed5e6dc4a5f958bc7cf1a68a26c) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:29.193725 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fb3ad-b96a-45f2-897a-4b7ba83e8d96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:29.195149 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949fb3ad-b96a-45f2-897a-4b7ba83e8d96 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:29.202942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 608/1219] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:35:29 2022] GET /identity/v3/role_assignments?user.id=93630ed5e6dc4a5f958bc7cf1a68a26c&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=64c2d69f15ca458c8267c1acfd42d9fc => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:31.381900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf05f9-c494-4ab1-9111-c78c9cb62531 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:31.382421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf05f9-c494-4ab1-9111-c78c9cb62531 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:31.382421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf05f9-c494-4ab1-9111-c78c9cb62531 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:31.383810 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 612/1220] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:35:31.389740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa802f-1460-4929-8ef7-c3ba643be978 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:31.390176 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa802f-1460-4929-8ef7-c3ba643be978 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:31.390329 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa802f-1460-4929-8ef7-c3ba643be978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:31.409025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-e5aa802f-1460-4929-8ef7-c3ba643be978 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:35:31.410512 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e5aa802f-1460-4929-8ef7-c3ba643be978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:35:31.484082 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 609/1221] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:35:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:35:31.509320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf05f9-c494-4ab1-9111-c78c9cb62531 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:31.615882 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf05f9-c494-4ab1-9111-c78c9cb62531 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:31.658282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f42b8b-bd9b-43b7-920b-4ceaa16092eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:31.659096 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f42b8b-bd9b-43b7-920b-4ceaa16092eb None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:31.659429 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f42b8b-bd9b-43b7-920b-4ceaa16092eb None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:31.659691 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f42b8b-bd9b-43b7-920b-4ceaa16092eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:31.671213 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f42b8b-bd9b-43b7-920b-4ceaa16092eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:35:31.679389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-58f42b8b-bd9b-43b7-920b-4ceaa16092eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:35:31.679389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:31.679389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:31.679389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:31.679389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:31.679389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:31.679389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:31.679389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:31.679389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:31.679389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:31.679389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:31.679389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:31.679389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:35:31.679389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:35:31.679389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:35:31.679389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:35:31.680452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:35:31.680452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:35:31.680452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:35:31.680452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:35:31.680452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:31.680452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:31.680452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:31.680452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:31.680452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:31.680452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:31.680452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:31.680452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:31.680452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:31.680452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:31.680452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:31.680452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:31.681412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:31.681412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:31.681412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:31.681412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:31.681412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:35:31.681412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:35:31.681412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:35:31.681412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:35:31.681412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:35:31.681412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:35:31.681412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:35:31.681412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:31.681412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 613/1222] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:35:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:31.687558 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa802f-1460-4929-8ef7-c3ba643be978 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:31.794179 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa802f-1460-4929-8ef7-c3ba643be978 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:31.818983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:31.832983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac87f8b-c420-435c-add5-7ddb8e8cf7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:31.833763 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac87f8b-c420-435c-add5-7ddb8e8cf7e5 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:31.834030 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac87f8b-c420-435c-add5-7ddb8e8cf7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:31.834390 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac87f8b-c420-435c-add5-7ddb8e8cf7e5 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:31.835155 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac87f8b-c420-435c-add5-7ddb8e8cf7e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:31.835562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac87f8b-c420-435c-add5-7ddb8e8cf7e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:31.836965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac87f8b-c420-435c-add5-7ddb8e8cf7e5 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:31.847077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 610/1223] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:35:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:31.853175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f42b8b-bd9b-43b7-920b-4ceaa16092eb None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:31.900820 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:31.969481 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f42b8b-bd9b-43b7-920b-4ceaa16092eb None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:32.007055 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d186abc-1c76-4dd3-9e77-66aab81c327f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:32.008059 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d186abc-1c76-4dd3-9e77-66aab81c327f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:32.008441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d186abc-1c76-4dd3-9e77-66aab81c327f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:32.008830 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d186abc-1c76-4dd3-9e77-66aab81c327f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:32.019874 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d186abc-1c76-4dd3-9e77-66aab81c327f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:32.021334 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d186abc-1c76-4dd3-9e77-66aab81c327f None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:32.040086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-1d186abc-1c76-4dd3-9e77-66aab81c327f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:32.040086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:32.040086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:32.040086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:32.040086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:32.040086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:32.040086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:32.040086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:32.040086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:32.040086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:32.040086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:32.040086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:32.040086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:35:32.040086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:35:32.040086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:35:32.041282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:35:32.041282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:32.041282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:32.041282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:32.041282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:32.041282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:32.041282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:32.041282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:32.041282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:32.041282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:32.041282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:32.041282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:32.041282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:32.041282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:32.041282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:32.041282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:32.041282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:32.042366 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:35:32.042366 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:35:32.042366 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:32.042366 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:32.042366 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 614/1224] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:32.048333 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac87f8b-c420-435c-add5-7ddb8e8cf7e5 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:32.155544 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac87f8b-c420-435c-add5-7ddb8e8cf7e5 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:32.189937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b96a1-d91a-4357-a57d-098b3ddc2fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:32.190755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b96a1-d91a-4357-a57d-098b3ddc2fd5 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:32.191041 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b96a1-d91a-4357-a57d-098b3ddc2fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:32.191324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b96a1-d91a-4357-a57d-098b3ddc2fd5 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:32.192205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b96a1-d91a-4357-a57d-098b3ddc2fd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:32.192543 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b96a1-d91a-4357-a57d-098b3ddc2fd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:32.194142 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b96a1-d91a-4357-a57d-098b3ddc2fd5 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:32.208936 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 611/1225] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:35:32.214759 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d186abc-1c76-4dd3-9e77-66aab81c327f None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:32.316700 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:32.324859 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d186abc-1c76-4dd3-9e77-66aab81c327f None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:32.363639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215704e1-6a64-49df-bf6a-f6aa02d5ed8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:32.364504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215704e1-6a64-49df-bf6a-f6aa02d5ed8f None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:32.364910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215704e1-6a64-49df-bf6a-f6aa02d5ed8f None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:32.365248 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215704e1-6a64-49df-bf6a-f6aa02d5ed8f None admin] PATH_INFO: `/v3/users/placement` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:32.374484 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215704e1-6a64-49df-bf6a-f6aa02d5ed8f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:32.376368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215704e1-6a64-49df-bf6a-f6aa02d5ed8f None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:32.394583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-215704e1-6a64-49df-bf6a-f6aa02d5ed8f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 22 11:35:32.394583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:32.394583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:32.394583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:32.394583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:32.394583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:32.394583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:32.394583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:32.394583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:32.394583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:32.394583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:32.394583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:32.394583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:35:32.394583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:35:32.394583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:35:32.394583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:35:32.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:32.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:32.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:35:32.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:35:32.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:35:32.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:35:32.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:32.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:32.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:32.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:32.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:32.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:32.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:32.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:32.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:32.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:32.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:32.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:32.397547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:32.397547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:32.397547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:35:32.397547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:35:32.397547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:35:32.397547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:35:32.397547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:35:32.397547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:35:32.397547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 22 11:35:32.397547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:32.397547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 615/1226] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:32.402220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b96a1-d91a-4357-a57d-098b3ddc2fd5 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:32.455407 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:32.516936 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b96a1-d91a-4357-a57d-098b3ddc2fd5 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:32.558760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78ada5-de20-4c34-9d94-544538826329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:32.559463 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78ada5-de20-4c34-9d94-544538826329 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:32.559747 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78ada5-de20-4c34-9d94-544538826329 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:32.560082 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78ada5-de20-4c34-9d94-544538826329 None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:32.560918 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78ada5-de20-4c34-9d94-544538826329 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:32.561245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78ada5-de20-4c34-9d94-544538826329 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:32.562709 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78ada5-de20-4c34-9d94-544538826329 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:32.580440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 612/1227] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jun 22 11:35:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:32.586526 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215704e1-6a64-49df-bf6a-f6aa02d5ed8f None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:32.706516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215704e1-6a64-49df-bf6a-f6aa02d5ed8f None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:32.745816 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4b156-62bf-4711-8147-e98b6a322d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:32.747142 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4b156-62bf-4711-8147-e98b6a322d54 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:32.747450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4b156-62bf-4711-8147-e98b6a322d54 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:32.747732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4b156-62bf-4711-8147-e98b6a322d54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:32.760412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b4b156-62bf-4711-8147-e98b6a322d54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:32.762188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b4b156-62bf-4711-8147-e98b6a322d54 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:32.780745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-44b4b156-62bf-4711-8147-e98b6a322d54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:32.780745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:32.780745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:32.780745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:32.780745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:32.780745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:32.780745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:32.780745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:32.780745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:32.780745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:32.780745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:32.780745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:32.780745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:35:32.780745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:35:32.780745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:35:32.781958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:35:32.781958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:32.781958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:32.781958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:32.781958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:32.781958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:32.781958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:32.781958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:32.781958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:32.781958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:32.781958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:32.781958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:32.781958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:32.781958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:32.781958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:32.781958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:32.781958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:32.783088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:35:32.783088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:35:32.783088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:32.783088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:32.783088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 616/1228] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:32.788074 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78ada5-de20-4c34-9d94-544538826329 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:32.906613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78ada5-de20-4c34-9d94-544538826329 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:32.910728 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:32.949132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bff06-5e31-4ae7-9678-fa5a1fa69c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:32.950325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bff06-5e31-4ae7-9678-fa5a1fa69c38 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:32.950601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bff06-5e31-4ae7-9678-fa5a1fa69c38 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:32.950875 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bff06-5e31-4ae7-9678-fa5a1fa69c38 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:32.951621 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bff06-5e31-4ae7-9678-fa5a1fa69c38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:32.951924 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bff06-5e31-4ae7-9678-fa5a1fa69c38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:32.953283 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bff06-5e31-4ae7-9678-fa5a1fa69c38 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:32.966699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 613/1229] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:32.972522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4b156-62bf-4711-8147-e98b6a322d54 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:32.983180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:33.089087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4b156-62bf-4711-8147-e98b6a322d54 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:33.127888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97854eca-b28d-4835-a080-30c6d80b3b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:33.128676 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97854eca-b28d-4835-a080-30c6d80b3b84 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:33.129078 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97854eca-b28d-4835-a080-30c6d80b3b84 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:33.129239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97854eca-b28d-4835-a080-30c6d80b3b84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:33.136549 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97854eca-b28d-4835-a080-30c6d80b3b84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:33.138418 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97854eca-b28d-4835-a080-30c6d80b3b84 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:33.154436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-97854eca-b28d-4835-a080-30c6d80b3b84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:35:33.154436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:33.154436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:33.154436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:33.154436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:33.154436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:33.154436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:33.154436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:33.154436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:33.154436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:33.154436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:33.154436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:33.154436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:35:33.154436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:35:33.154436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:35:33.155598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:35:33.155598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:33.155598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:33.155598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:33.155598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:33.155598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:33.155598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:33.155598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:33.155598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:33.155598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:33.155598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:33.155598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:33.155598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:33.155598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:33.155598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:33.155598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:33.155598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:33.156721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:35:33.156721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:35:33.156721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:35:33.156721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:35:33.156721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:35:33.156721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:35:33.156721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:35:33.156721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:33.156721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 617/1230] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:35:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:33.162288 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bff06-5e31-4ae7-9678-fa5a1fa69c38 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:33.274444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bff06-5e31-4ae7-9678-fa5a1fa69c38 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:33.310527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db540a7-75df-4b8c-9c00-1734101ac7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:33.311231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db540a7-75df-4b8c-9c00-1734101ac7fe None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:33.311555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db540a7-75df-4b8c-9c00-1734101ac7fe None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:33.311834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db540a7-75df-4b8c-9c00-1734101ac7fe None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:33.312658 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db540a7-75df-4b8c-9c00-1734101ac7fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:33.313015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db540a7-75df-4b8c-9c00-1734101ac7fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:33.314493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db540a7-75df-4b8c-9c00-1734101ac7fe None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:33.329196 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 614/1231] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:35:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:33.335671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97854eca-b28d-4835-a080-30c6d80b3b84 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:33.416003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:33.458032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97854eca-b28d-4835-a080-30c6d80b3b84 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:33.499127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5ca64-13c6-49e6-8bc8-bfe4c0f5ede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:33.500029 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5ca64-13c6-49e6-8bc8-bfe4c0f5ede2 None admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:33.500307 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5ca64-13c6-49e6-8bc8-bfe4c0f5ede2 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:33.500582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5ca64-13c6-49e6-8bc8-bfe4c0f5ede2 None admin] PATH_INFO: `/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/93630ed5e6dc4a5f958bc7cf1a68a26c/roles/64c2d69f15ca458c8267c1acfd42d9fc` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:33.518261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea5ca64-13c6-49e6-8bc8-bfe4c0f5ede2 None admin] RBAC: Authorizing `identity:create_grant(project_id=47d2bb094c9843c9a7726f1d672d2433, user_id=93630ed5e6dc4a5f958bc7cf1a68a26c, role_id=64c2d69f15ca458c8267c1acfd42d9fc)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:33.520130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea5ca64-13c6-49e6-8bc8-bfe4c0f5ede2 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:33.535985 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 618/1232] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:35:33 2022] PUT /identity/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/93630ed5e6dc4a5f958bc7cf1a68a26c/roles/64c2d69f15ca458c8267c1acfd42d9fc => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:35:35.624189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b1fee-e595-413c-8bbe-9bfe40ad291d None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:35.624784 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b1fee-e595-413c-8bbe-9bfe40ad291d None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:35.624784 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b1fee-e595-413c-8bbe-9bfe40ad291d None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:35.626124 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 615/1233] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:35:35.631860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6e93d-b440-4719-ba07-542ebf182204 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:35.632305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6e93d-b440-4719-ba07-542ebf182204 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:35.632501 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6e93d-b440-4719-ba07-542ebf182204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:35.650618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-43a6e93d-b440-4719-ba07-542ebf182204 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:35:35.651950 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-43a6e93d-b440-4719-ba07-542ebf182204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:35:35.722963 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 619/1234] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:35:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:35:35.747742 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b1fee-e595-413c-8bbe-9bfe40ad291d None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:35.775152 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:35.870989 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b1fee-e595-413c-8bbe-9bfe40ad291d None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:35.908263 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40884d47-1179-4579-9c4f-72b803a36dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:35.909055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40884d47-1179-4579-9c4f-72b803a36dbe None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:35.909341 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40884d47-1179-4579-9c4f-72b803a36dbe None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:35.909616 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40884d47-1179-4579-9c4f-72b803a36dbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:35.925110 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40884d47-1179-4579-9c4f-72b803a36dbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:35:35.933447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-40884d47-1179-4579-9c4f-72b803a36dbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:35:35.933447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:35.933447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:35.933447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:35.933447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:35.933447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:35.933447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:35.933447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:35.933447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:35.933447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:35.933447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:35.933447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:35.933447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:35:35.933447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:35:35.933447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:35:35.933447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:35:35.934511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:35:35.934511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:35:35.934511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:35:35.934511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:35:35.934511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:35.934511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:35.934511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:35.934511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:35.934511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:35.934511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:35.934511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:35.934511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:35.934511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:35.934511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:35.934511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:35.934511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:35.935415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:35.935415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:35.935415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:35.935415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:35.935415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:35:35.935415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:35:35.935415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:35:35.935415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:35:35.935415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:35:35.935415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:35:35.935415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 22 11:35:35.935415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:35:35.935415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 616/1235] 10.222.0.36 () {56 vars in 1102 bytes} [Wed Jun 22 11:35:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 22 11:35:35.942403 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6e93d-b440-4719-ba07-542ebf182204 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:35.998371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:36.072725 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6e93d-b440-4719-ba07-542ebf182204 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:36.116242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764e5c2-1da7-41a4-8b0e-bde02e0ecdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:36.117144 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764e5c2-1da7-41a4-8b0e-bde02e0ecdb2 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:36.117479 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764e5c2-1da7-41a4-8b0e-bde02e0ecdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:36.117785 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764e5c2-1da7-41a4-8b0e-bde02e0ecdb2 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:36.118572 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764e5c2-1da7-41a4-8b0e-bde02e0ecdb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:36.118881 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764e5c2-1da7-41a4-8b0e-bde02e0ecdb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:36.120289 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764e5c2-1da7-41a4-8b0e-bde02e0ecdb2 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:36.129527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 620/1236] 10.222.0.36 () {56 vars in 1103 bytes} [Wed Jun 22 11:35:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:36.135725 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40884d47-1179-4579-9c4f-72b803a36dbe None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:36.229835 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:36.247543 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40884d47-1179-4579-9c4f-72b803a36dbe None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:36.285706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbb941-2dde-4d99-b2d9-8890f3741fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:36.286517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbb941-2dde-4d99-b2d9-8890f3741fd7 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:36.286801 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbb941-2dde-4d99-b2d9-8890f3741fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:36.287072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbb941-2dde-4d99-b2d9-8890f3741fd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:36.298930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbb941-2dde-4d99-b2d9-8890f3741fd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:36.300888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbb941-2dde-4d99-b2d9-8890f3741fd7 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:36.321239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-8bdbb941-2dde-4d99-b2d9-8890f3741fd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:36.321239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:36.321239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:36.321239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:36.321239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:36.321239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:36.321239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:36.321239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:36.321239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:36.321239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:36.321239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:36.321239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:36.321239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:35:36.321239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:35:36.321239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:35:36.322385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:35:36.322385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:36.322385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:36.322385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:36.322385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:36.322385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:36.322385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:36.322385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:36.322385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:36.322385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:36.322385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:36.322385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:36.322385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:36.322385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:36.322385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:36.322385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:36.322385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:36.329279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:35:36.329279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:35:36.329279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:36.329279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:35:36.329279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 617/1237] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:36.329823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764e5c2-1da7-41a4-8b0e-bde02e0ecdb2 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:36.445982 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764e5c2-1da7-41a4-8b0e-bde02e0ecdb2 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:36.455925 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:36.485070 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fd0f6-469d-46f7-9785-9e332f77da79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:36.485694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fd0f6-469d-46f7-9785-9e332f77da79 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:36.485987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fd0f6-469d-46f7-9785-9e332f77da79 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:36.486263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fd0f6-469d-46f7-9785-9e332f77da79 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:36.486991 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fd0f6-469d-46f7-9785-9e332f77da79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:36.487300 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fd0f6-469d-46f7-9785-9e332f77da79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:36.488710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fd0f6-469d-46f7-9785-9e332f77da79 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:36.500946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 621/1238] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:35:36.507313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbb941-2dde-4d99-b2d9-8890f3741fd7 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:36.627609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbb941-2dde-4d99-b2d9-8890f3741fd7 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:36.669602 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126782f-6191-4382-a45d-041cd628910d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:36.670529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126782f-6191-4382-a45d-041cd628910d None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:36.670781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126782f-6191-4382-a45d-041cd628910d None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:36.671108 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126782f-6191-4382-a45d-041cd628910d None admin] PATH_INFO: `/v3/users/placement` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:36.674649 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:36.681775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126782f-6191-4382-a45d-041cd628910d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:36.684140 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126782f-6191-4382-a45d-041cd628910d None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:36.700097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-9126782f-6191-4382-a45d-041cd628910d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 22 11:35:36.700097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:36.700097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:36.700097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:36.700097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:36.700097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:36.700097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:36.700097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:36.700097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:36.700097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:36.700097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:36.700097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:36.700097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:35:36.700097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:35:36.700097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:35:36.700097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:35:36.701358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:36.701358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:36.701358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:35:36.701358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:35:36.701358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:35:36.701358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:35:36.701358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:36.701358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:36.701358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:36.701358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:36.701358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:36.701358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:36.701358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:36.701358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:36.701358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:36.701358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:36.701358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:36.701358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:36.702609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:36.702609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:36.702609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:35:36.702609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:35:36.702609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:35:36.702609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:35:36.702609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:35:36.702609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:35:36.702609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 22 11:35:36.702609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:35:36.702609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 618/1239] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:36.707706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fd0f6-469d-46f7-9785-9e332f77da79 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:36.822434 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fd0f6-469d-46f7-9785-9e332f77da79 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:36.859980 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa2aae-fb8e-43d4-bce2-a7766e4cfd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:36.860772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa2aae-fb8e-43d4-bce2-a7766e4cfd3e None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:36.861065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa2aae-fb8e-43d4-bce2-a7766e4cfd3e None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:36.861345 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa2aae-fb8e-43d4-bce2-a7766e4cfd3e None admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:36.862192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa2aae-fb8e-43d4-bce2-a7766e4cfd3e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:36.862502 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa2aae-fb8e-43d4-bce2-a7766e4cfd3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:36.863982 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa2aae-fb8e-43d4-bce2-a7766e4cfd3e None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:36.885729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 622/1240] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jun 22 11:35:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:36.891913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126782f-6191-4382-a45d-041cd628910d None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:37.014425 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126782f-6191-4382-a45d-041cd628910d None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:37.053666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ffe40-7cee-4b37-ad39-3095a27a8a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:37.054319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ffe40-7cee-4b37-ad39-3095a27a8a9b None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:37.054607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ffe40-7cee-4b37-ad39-3095a27a8a9b None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:37.054885 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ffe40-7cee-4b37-ad39-3095a27a8a9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:37.119381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:37.127175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ffe40-7cee-4b37-ad39-3095a27a8a9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:37.128798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ffe40-7cee-4b37-ad39-3095a27a8a9b None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:37.240046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 619/1241] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:35:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 22 11:35:37.241406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa2aae-fb8e-43d4-bce2-a7766e4cfd3e None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:37.272747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:37.354741 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa2aae-fb8e-43d4-bce2-a7766e4cfd3e None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:37.399299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c088e-efb3-4852-be2a-2687d832e718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:37.400580 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c088e-efb3-4852-be2a-2687d832e718 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:37.400986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c088e-efb3-4852-be2a-2687d832e718 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:37.401343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c088e-efb3-4852-be2a-2687d832e718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:37.415301 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c088e-efb3-4852-be2a-2687d832e718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:37.416765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c088e-efb3-4852-be2a-2687d832e718 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:37.436397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-d96c088e-efb3-4852-be2a-2687d832e718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:37.436397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:37.436397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:37.436397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:37.436397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:37.436397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:37.436397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:37.436397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:37.436397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:37.436397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:37.436397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:37.436397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:37.436397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:35:37.436397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:35:37.436397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:35:37.437395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:35:37.437395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:37.437395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:37.437395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:37.437395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:37.437395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:37.437395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:37.437395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:37.437395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:37.437395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:37.437395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:37.437395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:37.437395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:37.437395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:37.437395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:37.437395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:37.437395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:37.438312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:35:37.438312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:35:37.438312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:37.438312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:37.438312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 623/1242] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:37.443702 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ffe40-7cee-4b37-ad39-3095a27a8a9b None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:37.553064 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:37.561214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ffe40-7cee-4b37-ad39-3095a27a8a9b None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:37.599913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9922b-82b0-4558-9378-69290dc9f928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:37.600588 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9922b-82b0-4558-9378-69290dc9f928 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:37.600866 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9922b-82b0-4558-9378-69290dc9f928 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:37.601138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9922b-82b0-4558-9378-69290dc9f928 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:37.601881 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9922b-82b0-4558-9378-69290dc9f928 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:37.602179 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9922b-82b0-4558-9378-69290dc9f928 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:37.603550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9922b-82b0-4558-9378-69290dc9f928 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:37.615667 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 620/1243] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:37.621879 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c088e-efb3-4852-be2a-2687d832e718 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:37.717054 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:37.746231 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c088e-efb3-4852-be2a-2687d832e718 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:37.785911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd928900-0327-4316-9581-9f797bb22bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:37.786998 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd928900-0327-4316-9581-9f797bb22bc5 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:37.787218 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd928900-0327-4316-9581-9f797bb22bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:37.787547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd928900-0327-4316-9581-9f797bb22bc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:37.795770 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd928900-0327-4316-9581-9f797bb22bc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:37.797329 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd928900-0327-4316-9581-9f797bb22bc5 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:37.816740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-bd928900-0327-4316-9581-9f797bb22bc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:35:37.816740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:37.816740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:37.816740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:37.816740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:37.816740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:37.816740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:37.816740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:37.816740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:37.816740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:37.816740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:37.816740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:37.816740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:35:37.816740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:35:37.816740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:35:37.818030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:35:37.818030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:37.818030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:37.818030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:37.818030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:37.818030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:37.818030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:37.818030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:37.818030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:37.818030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:37.818030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:37.818030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:37.818030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:37.818030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:37.818030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:37.818030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:37.818030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:37.818937 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:35:37.818937 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:35:37.818937 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:35:37.818937 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:35:37.818937 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:35:37.818937 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:35:37.818937 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:35:37.818937 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:37.818937 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 624/1244] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:35:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:37.824191 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9922b-82b0-4558-9378-69290dc9f928 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:37.931247 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9922b-82b0-4558-9378-69290dc9f928 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:37.970777 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba1eb5-9fb7-4f8d-97ae-5fbd0d827f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:37.971528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba1eb5-9fb7-4f8d-97ae-5fbd0d827f44 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:37.971824 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba1eb5-9fb7-4f8d-97ae-5fbd0d827f44 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:37.972109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba1eb5-9fb7-4f8d-97ae-5fbd0d827f44 None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:37.973009 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba1eb5-9fb7-4f8d-97ae-5fbd0d827f44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:37.973341 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba1eb5-9fb7-4f8d-97ae-5fbd0d827f44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:37.975026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba1eb5-9fb7-4f8d-97ae-5fbd0d827f44 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:37.990224 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 621/1245] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:35:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:37.999290 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd928900-0327-4316-9581-9f797bb22bc5 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:38.110138 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd928900-0327-4316-9581-9f797bb22bc5 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:38.146843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825723e7-bd54-4827-a122-007680ae2f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:38.147539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825723e7-bd54-4827-a122-007680ae2f8c None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:38.147823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825723e7-bd54-4827-a122-007680ae2f8c None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:38.148101 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825723e7-bd54-4827-a122-007680ae2f8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:38.148930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825723e7-bd54-4827-a122-007680ae2f8c None admin] RBAC: Adding query filter params (role.id=64c2d69f15ca458c8267c1acfd42d9fc, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=93630ed5e6dc4a5f958bc7cf1a68a26c) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:38.149229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825723e7-bd54-4827-a122-007680ae2f8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:38.150783 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825723e7-bd54-4827-a122-007680ae2f8c None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:38.159680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 625/1246] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:35:37 2022] GET /identity/v3/role_assignments?user.id=93630ed5e6dc4a5f958bc7cf1a68a26c&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=64c2d69f15ca458c8267c1acfd42d9fc => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:40.454934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84552287-360e-42eb-96ea-201e2d604594 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:40.455591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84552287-360e-42eb-96ea-201e2d604594 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:40.455591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84552287-360e-42eb-96ea-201e2d604594 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:40.456925 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 622/1247] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:35:40.462925 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236ca34-61fb-4c74-8833-8cb965961ac5 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:40.463655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236ca34-61fb-4c74-8833-8cb965961ac5 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:40.463655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236ca34-61fb-4c74-8833-8cb965961ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:40.469638 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:40.482564 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-5236ca34-61fb-4c74-8833-8cb965961ac5 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:35:40.483985 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5236ca34-61fb-4c74-8833-8cb965961ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:35:40.567086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 626/1248] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:35:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:35:40.592794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84552287-360e-42eb-96ea-201e2d604594 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:40.613074 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:40.718258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84552287-360e-42eb-96ea-201e2d604594 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:40.759079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc66fc-acb5-48ff-945c-c3f5f4ce2fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:40.760091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc66fc-acb5-48ff-945c-c3f5f4ce2fcf None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:40.760628 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc66fc-acb5-48ff-945c-c3f5f4ce2fcf None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:40.760891 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc66fc-acb5-48ff-945c-c3f5f4ce2fcf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:40.772691 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc66fc-acb5-48ff-945c-c3f5f4ce2fcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:35:40.781811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-b7bc66fc-acb5-48ff-945c-c3f5f4ce2fcf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:35:40.781811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:40.781811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:40.781811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:40.781811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:40.781811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:40.781811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:40.781811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:40.781811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:40.781811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:40.781811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:40.781811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:40.781811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:35:40.781811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:35:40.781811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:35:40.781811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:35:40.783121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:35:40.783121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:35:40.783121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:35:40.783121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:35:40.783121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:40.783121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:40.783121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:40.783121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:40.783121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:40.783121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:40.783121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:40.783121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:40.783121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:40.783121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:40.783121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:40.783121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:40.784214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:40.784214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:40.784214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:40.784214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:40.784214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:35:40.784214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:35:40.784214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:35:40.784214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:35:40.784214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:35:40.784214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:35:40.784214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:35:40.784214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:35:40.784214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 623/1249] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:35:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:40.789968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236ca34-61fb-4c74-8833-8cb965961ac5 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:40.909817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236ca34-61fb-4c74-8833-8cb965961ac5 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:40.948054 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:40.954369 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90072ac7-e316-4f42-8e22-bf4207d84c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:40.955123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90072ac7-e316-4f42-8e22-bf4207d84c74 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:40.955457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90072ac7-e316-4f42-8e22-bf4207d84c74 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:40.955707 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90072ac7-e316-4f42-8e22-bf4207d84c74 None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:40.956574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90072ac7-e316-4f42-8e22-bf4207d84c74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:40.956930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90072ac7-e316-4f42-8e22-bf4207d84c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:40.958586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90072ac7-e316-4f42-8e22-bf4207d84c74 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:40.974929 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 627/1250] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:35:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:40.976728 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc66fc-acb5-48ff-945c-c3f5f4ce2fcf None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:41.072450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:41.101057 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc66fc-acb5-48ff-945c-c3f5f4ce2fcf None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:41.142101 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e4043-0a05-4000-8fb5-540ad45d3a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:41.142906 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e4043-0a05-4000-8fb5-540ad45d3a57 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:41.143193 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e4043-0a05-4000-8fb5-540ad45d3a57 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:41.143524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e4043-0a05-4000-8fb5-540ad45d3a57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:41.156285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e4043-0a05-4000-8fb5-540ad45d3a57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:41.157766 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e4043-0a05-4000-8fb5-540ad45d3a57 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:41.177298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-dc8e4043-0a05-4000-8fb5-540ad45d3a57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:41.177298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:41.177298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:41.177298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:41.177298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:41.177298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:41.177298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:41.177298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:41.177298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:41.177298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:41.177298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:41.177298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:41.177298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:35:41.177298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:35:41.177298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:35:41.178553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:35:41.178553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:41.178553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:41.178553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:41.178553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:41.178553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:41.178553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:41.178553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:41.178553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:41.178553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:41.178553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:41.178553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:41.178553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:41.178553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:41.178553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:41.178553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:41.178553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:41.179561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:35:41.179561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:35:41.179561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:41.179561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:35:41.179561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 624/1251] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:41.185792 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90072ac7-e316-4f42-8e22-bf4207d84c74 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:41.310081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90072ac7-e316-4f42-8e22-bf4207d84c74 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:41.347729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc4b7e-0eba-4dd3-9b98-f0dc495faa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:41.348516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc4b7e-0eba-4dd3-9b98-f0dc495faa21 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:41.348751 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc4b7e-0eba-4dd3-9b98-f0dc495faa21 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:41.349025 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc4b7e-0eba-4dd3-9b98-f0dc495faa21 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:41.349829 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc4b7e-0eba-4dd3-9b98-f0dc495faa21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:41.350102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc4b7e-0eba-4dd3-9b98-f0dc495faa21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:41.351591 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc4b7e-0eba-4dd3-9b98-f0dc495faa21 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:41.365325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 628/1252] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:35:41.371523 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e4043-0a05-4000-8fb5-540ad45d3a57 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:41.491793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e4043-0a05-4000-8fb5-540ad45d3a57 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:41.528101 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:41.537433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3d7a4-abe2-4954-91f4-8dc0d4356633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:41.538357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3d7a4-abe2-4954-91f4-8dc0d4356633 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:41.538641 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3d7a4-abe2-4954-91f4-8dc0d4356633 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:41.538962 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3d7a4-abe2-4954-91f4-8dc0d4356633 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:41.547680 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3d7a4-abe2-4954-91f4-8dc0d4356633 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:41.549129 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3d7a4-abe2-4954-91f4-8dc0d4356633 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:41.565077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-41f3d7a4-abe2-4954-91f4-8dc0d4356633 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 22 11:35:41.565077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:41.565077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:41.565077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:41.565077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:41.565077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:41.565077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:41.565077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:41.565077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:41.565077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:41.565077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:41.565077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:41.565077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:35:41.565077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:35:41.565077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:35:41.565077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:35:41.566278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:41.566278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:41.566278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:35:41.566278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:35:41.566278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:35:41.566278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:35:41.566278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:41.566278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:41.566278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:41.566278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:41.566278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:41.566278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:41.566278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:41.566278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:41.566278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:41.566278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:41.566278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:41.566278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:41.567241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:41.567241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:41.567241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:35:41.567241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:35:41.567241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:35:41.567241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:35:41.567241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:35:41.567241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:35:41.567241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 22 11:35:41.567241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:35:41.567241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 625/1253] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:41.572171 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc4b7e-0eba-4dd3-9b98-f0dc495faa21 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:41.637877 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:41.697217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc4b7e-0eba-4dd3-9b98-f0dc495faa21 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:41.735054 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c15f9f-1811-48cb-a577-2c9cabbd7086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:41.735760 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c15f9f-1811-48cb-a577-2c9cabbd7086 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:41.736111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c15f9f-1811-48cb-a577-2c9cabbd7086 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:41.736467 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c15f9f-1811-48cb-a577-2c9cabbd7086 None admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:41.737318 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c15f9f-1811-48cb-a577-2c9cabbd7086 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:41.737645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c15f9f-1811-48cb-a577-2c9cabbd7086 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:41.739079 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c15f9f-1811-48cb-a577-2c9cabbd7086 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:41.760536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 629/1254] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jun 22 11:35:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:35:41.767374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3d7a4-abe2-4954-91f4-8dc0d4356633 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:41.895466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3d7a4-abe2-4954-91f4-8dc0d4356633 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:41.932884 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0360dbb-6041-4f3c-86bf-ea38acb4c64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:41.933668 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0360dbb-6041-4f3c-86bf-ea38acb4c64f None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:41.933934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0360dbb-6041-4f3c-86bf-ea38acb4c64f None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:41.934269 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0360dbb-6041-4f3c-86bf-ea38acb4c64f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:41.999784 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:42.014115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0360dbb-6041-4f3c-86bf-ea38acb4c64f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:42.015827 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0360dbb-6041-4f3c-86bf-ea38acb4c64f None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:42.129871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 626/1255] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:35:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 22 11:35:42.135627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c15f9f-1811-48cb-a577-2c9cabbd7086 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:42.241626 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c15f9f-1811-48cb-a577-2c9cabbd7086 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:42.257135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:42.280595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347cc7e-7b3e-48cc-a497-6e1aae410ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:42.281391 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347cc7e-7b3e-48cc-a497-6e1aae410ed3 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:42.281672 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347cc7e-7b3e-48cc-a497-6e1aae410ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:42.283282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347cc7e-7b3e-48cc-a497-6e1aae410ed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:42.293267 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347cc7e-7b3e-48cc-a497-6e1aae410ed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:42.294764 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347cc7e-7b3e-48cc-a497-6e1aae410ed3 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:42.314396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-0347cc7e-7b3e-48cc-a497-6e1aae410ed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:42.314396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:42.314396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:42.314396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:42.314396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:42.314396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:42.314396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:42.314396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:42.314396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:42.314396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:42.314396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:42.314396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:42.314396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:35:42.314396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:35:42.314396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:35:42.315604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:35:42.315604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:42.315604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:42.315604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:42.315604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:42.315604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:42.315604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:42.315604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:42.315604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:42.315604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:42.315604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:42.315604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:42.315604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:42.315604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:42.315604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:42.315604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:42.315604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:42.316710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:35:42.316710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:35:42.316710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:42.316710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:42.318303 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 630/1256] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:42.324223 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0360dbb-6041-4f3c-86bf-ea38acb4c64f None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:42.420610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:42.435083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0360dbb-6041-4f3c-86bf-ea38acb4c64f None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:42.471312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a818b60-ecde-40f3-b5a0-7d10dbce29ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:42.471998 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a818b60-ecde-40f3-b5a0-7d10dbce29ac None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:42.472282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a818b60-ecde-40f3-b5a0-7d10dbce29ac None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:42.472552 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a818b60-ecde-40f3-b5a0-7d10dbce29ac None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:42.473263 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a818b60-ecde-40f3-b5a0-7d10dbce29ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:42.473567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a818b60-ecde-40f3-b5a0-7d10dbce29ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:42.474919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a818b60-ecde-40f3-b5a0-7d10dbce29ac None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:42.487377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 627/1257] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:42.493227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347cc7e-7b3e-48cc-a497-6e1aae410ed3 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:42.602498 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347cc7e-7b3e-48cc-a497-6e1aae410ed3 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:42.645549 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1429d-2f9e-4a18-8fed-44d17e9e9b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:42.646583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1429d-2f9e-4a18-8fed-44d17e9e9b2e None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:42.646942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1429d-2f9e-4a18-8fed-44d17e9e9b2e None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:42.647311 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1429d-2f9e-4a18-8fed-44d17e9e9b2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:42.657178 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1429d-2f9e-4a18-8fed-44d17e9e9b2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:42.659049 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1429d-2f9e-4a18-8fed-44d17e9e9b2e None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:42.677974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-81b1429d-2f9e-4a18-8fed-44d17e9e9b2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:35:42.677974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:42.677974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:42.677974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:42.677974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:42.677974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:42.677974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:42.677974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:42.677974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:42.677974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:42.677974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:42.677974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:42.677974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:35:42.677974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:35:42.677974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:35:42.679343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:35:42.679343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:42.679343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:42.679343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:42.679343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:42.679343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:42.679343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:42.679343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:42.679343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:42.679343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:42.679343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:42.679343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:42.679343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:42.679343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:42.679343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:42.679343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:42.679343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:42.680445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:35:42.680445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:35:42.680445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:35:42.680445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:35:42.680445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:35:42.680445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:35:42.680445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:35:42.680445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:42.680445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 631/1258] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:35:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:42.685868 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a818b60-ecde-40f3-b5a0-7d10dbce29ac None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:42.799637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a818b60-ecde-40f3-b5a0-7d10dbce29ac None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:42.842709 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d521e-3513-4781-971e-765d9c0d54bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:42.843407 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d521e-3513-4781-971e-765d9c0d54bb None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:42.843698 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d521e-3513-4781-971e-765d9c0d54bb None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:42.843977 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d521e-3513-4781-971e-765d9c0d54bb None admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:42.844775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d521e-3513-4781-971e-765d9c0d54bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:42.845087 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d521e-3513-4781-971e-765d9c0d54bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:42.846514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d521e-3513-4781-971e-765d9c0d54bb None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:42.860861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 628/1259] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:35:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:42.867395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1429d-2f9e-4a18-8fed-44d17e9e9b2e None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:42.895910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:42.984220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1429d-2f9e-4a18-8fed-44d17e9e9b2e None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:43.026131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f84ee2b-01f5-45b1-bac4-511eda7e2a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:43.027783 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84ee2b-01f5-45b1-bac4-511eda7e2a74 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:43.028159 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84ee2b-01f5-45b1-bac4-511eda7e2a74 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:43.028456 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f84ee2b-01f5-45b1-bac4-511eda7e2a74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:43.030399 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84ee2b-01f5-45b1-bac4-511eda7e2a74 None admin] RBAC: Adding query filter params (role.id=96c584428d924fdba9671949f760c406, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=93630ed5e6dc4a5f958bc7cf1a68a26c) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:43.030768 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84ee2b-01f5-45b1-bac4-511eda7e2a74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:43.032374 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f84ee2b-01f5-45b1-bac4-511eda7e2a74 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:43.039055 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 632/1260] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:35:42 2022] GET /identity/v3/role_assignments?user.id=93630ed5e6dc4a5f958bc7cf1a68a26c&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=96c584428d924fdba9671949f760c406 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:45.095150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a790a0-c97d-45eb-9906-91c069f058de None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:45.095725 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a790a0-c97d-45eb-9906-91c069f058de None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:45.095725 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a790a0-c97d-45eb-9906-91c069f058de None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:45.097099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 629/1261] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:35:45.102111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3f816-9fca-413e-b245-bd8f2196e184 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:45.102477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3f816-9fca-413e-b245-bd8f2196e184 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:45.102673 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3f816-9fca-413e-b245-bd8f2196e184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:45.119059 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-24d3f816-9fca-413e-b245-bd8f2196e184 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:35:45.120261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-24d3f816-9fca-413e-b245-bd8f2196e184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:35:45.184343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 633/1262] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:35:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:35:45.208149 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a790a0-c97d-45eb-9906-91c069f058de None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:45.219520 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:45.307112 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a790a0-c97d-45eb-9906-91c069f058de None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:45.342449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5267d-14a2-4df5-aca1-4b47bb034c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:45.343224 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5267d-14a2-4df5-aca1-4b47bb034c90 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:45.343533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5267d-14a2-4df5-aca1-4b47bb034c90 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:45.343805 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5267d-14a2-4df5-aca1-4b47bb034c90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:45.353793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5267d-14a2-4df5-aca1-4b47bb034c90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:35:45.361344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-4bf5267d-14a2-4df5-aca1-4b47bb034c90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:35:45.361344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:45.361344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:45.361344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:45.361344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:45.361344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:45.361344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:45.361344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:45.361344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:45.361344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:45.361344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:45.361344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:45.361344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:35:45.361344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:35:45.361344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:35:45.361344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:35:45.362427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:35:45.362427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:35:45.362427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:35:45.362427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:35:45.362427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:45.362427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:45.362427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:45.362427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:45.362427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:45.362427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:45.362427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:45.362427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:45.362427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:45.362427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:45.362427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:45.362427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:45.363488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:45.363488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:45.363488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:45.363488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:45.363488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:35:45.363488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:35:45.363488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:35:45.363488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:35:45.363488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:35:45.363488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:35:45.363488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:35:45.363488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:35:45.363488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 630/1263] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:35:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:45.370536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3f816-9fca-413e-b245-bd8f2196e184 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:45.393629 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:45.472078 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3f816-9fca-413e-b245-bd8f2196e184 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:45.504826 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee1863-5096-4257-ac36-0256a96fa08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:45.505496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee1863-5096-4257-ac36-0256a96fa08b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:45.505783 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee1863-5096-4257-ac36-0256a96fa08b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:45.506062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee1863-5096-4257-ac36-0256a96fa08b None admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:45.506811 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee1863-5096-4257-ac36-0256a96fa08b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:45.507120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee1863-5096-4257-ac36-0256a96fa08b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:45.508529 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee1863-5096-4257-ac36-0256a96fa08b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:45.517496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 634/1264] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:35:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:45.523129 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5267d-14a2-4df5-aca1-4b47bb034c90 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:45.588324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:45.623448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5267d-14a2-4df5-aca1-4b47bb034c90 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:45.659169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94605c72-79dc-4c60-9b20-4139cc8f3525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:45.660089 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94605c72-79dc-4c60-9b20-4139cc8f3525 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:45.660423 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94605c72-79dc-4c60-9b20-4139cc8f3525 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:45.660750 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94605c72-79dc-4c60-9b20-4139cc8f3525 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:45.672331 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94605c72-79dc-4c60-9b20-4139cc8f3525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:45.673850 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94605c72-79dc-4c60-9b20-4139cc8f3525 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:45.692459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-94605c72-79dc-4c60-9b20-4139cc8f3525 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:45.692459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:45.692459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:45.692459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:45.692459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:45.692459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:45.692459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:45.692459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:45.692459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:45.692459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:45.692459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:45.692459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:45.692459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:35:45.692459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:35:45.692459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:35:45.693540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:35:45.693540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:45.693540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:45.693540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:45.693540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:45.693540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:45.693540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:45.693540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:45.693540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:45.693540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:45.693540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:45.693540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:45.693540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:45.693540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:45.693540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:45.693540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:45.693540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:45.694854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:35:45.694854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:35:45.694854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:45.694854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:35:45.694854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 631/1265] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:45.699668 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee1863-5096-4257-ac36-0256a96fa08b None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:45.781377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:45.802065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee1863-5096-4257-ac36-0256a96fa08b None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:45.835146 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf4d8a-12fa-442c-af5c-60b6c7153db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:45.835833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf4d8a-12fa-442c-af5c-60b6c7153db2 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:45.836116 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf4d8a-12fa-442c-af5c-60b6c7153db2 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:45.836390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf4d8a-12fa-442c-af5c-60b6c7153db2 None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:45.837099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf4d8a-12fa-442c-af5c-60b6c7153db2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:45.837403 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf4d8a-12fa-442c-af5c-60b6c7153db2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:45.838765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf4d8a-12fa-442c-af5c-60b6c7153db2 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:45.850494 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 635/1266] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:45.856822 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94605c72-79dc-4c60-9b20-4139cc8f3525 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:45.954530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94605c72-79dc-4c60-9b20-4139cc8f3525 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:45.975325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:45.989668 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedb53a-ebb3-41ff-8a98-540b66852cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:45.990460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedb53a-ebb3-41ff-8a98-540b66852cfb None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:45.990745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedb53a-ebb3-41ff-8a98-540b66852cfb None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:45.991023 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedb53a-ebb3-41ff-8a98-540b66852cfb None admin] PATH_INFO: `/v3/users/placement` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:45.998328 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aedb53a-ebb3-41ff-8a98-540b66852cfb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:45.999794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aedb53a-ebb3-41ff-8a98-540b66852cfb None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:46.013853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-9aedb53a-ebb3-41ff-8a98-540b66852cfb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 22 11:35:46.013853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:46.013853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:46.013853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:46.013853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:46.013853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:46.013853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:46.013853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:46.013853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:46.013853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:46.013853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:46.013853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:46.013853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:35:46.013853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:35:46.013853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:35:46.013853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:35:46.015459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:46.015459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:46.015459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:35:46.015459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:35:46.015459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:35:46.015459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:35:46.015459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:46.015459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:46.015459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:46.015459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:46.015459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:46.015459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:46.015459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:46.015459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:46.015459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:46.015459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:46.015459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:46.015459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:46.016634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:46.016634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:46.016634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:35:46.016634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:35:46.016634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:35:46.016634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:35:46.016634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:35:46.016634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:35:46.016634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 22 11:35:46.016634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:35:46.016634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 632/1267] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:46.021016 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf4d8a-12fa-442c-af5c-60b6c7153db2 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:46.119790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf4d8a-12fa-442c-af5c-60b6c7153db2 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:46.152867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc9c03-074b-4759-98e2-b0ae6609858c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:46.153527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc9c03-074b-4759-98e2-b0ae6609858c None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:46.153809 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc9c03-074b-4759-98e2-b0ae6609858c None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:46.154087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc9c03-074b-4759-98e2-b0ae6609858c None admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:46.154909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc9c03-074b-4759-98e2-b0ae6609858c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:46.155215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc9c03-074b-4759-98e2-b0ae6609858c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:46.156666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc9c03-074b-4759-98e2-b0ae6609858c None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:46.164987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:46.174987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 636/1268] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jun 22 11:35:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:46.180591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedb53a-ebb3-41ff-8a98-540b66852cfb None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:46.275515 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedb53a-ebb3-41ff-8a98-540b66852cfb None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:46.308714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc31cce-9b52-4f80-8914-249720ca70bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:46.309481 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc31cce-9b52-4f80-8914-249720ca70bb None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:46.309767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc31cce-9b52-4f80-8914-249720ca70bb None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:46.310036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc31cce-9b52-4f80-8914-249720ca70bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:46.320105 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc31cce-9b52-4f80-8914-249720ca70bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:46.321547 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc31cce-9b52-4f80-8914-249720ca70bb None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:46.338300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-bdc31cce-9b52-4f80-8914-249720ca70bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:46.338300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:46.338300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:46.338300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:46.338300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:46.338300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:46.338300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:46.338300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:46.338300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:46.338300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:46.338300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:46.338300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:46.338300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:35:46.338300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:35:46.338300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:35:46.339438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:35:46.339438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:46.339438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:46.339438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:46.339438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:46.339438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:46.339438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:46.339438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:46.339438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:46.339438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:46.339438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:46.339438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:46.339438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:46.339438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:46.339438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:46.339438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:46.339438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:46.340696 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:35:46.340696 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:35:46.340696 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:46.340696 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:35:46.340696 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 633/1269] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:46.345365 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc9c03-074b-4759-98e2-b0ae6609858c None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:46.442659 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc9c03-074b-4759-98e2-b0ae6609858c None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:46.476278 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1a59c-c1bd-42d5-81c4-9d6559e49c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:46.476924 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1a59c-c1bd-42d5-81c4-9d6559e49c2c None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:46.477206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1a59c-c1bd-42d5-81c4-9d6559e49c2c None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:46.477486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1a59c-c1bd-42d5-81c4-9d6559e49c2c None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:46.478196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1a59c-c1bd-42d5-81c4-9d6559e49c2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:46.478501 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1a59c-c1bd-42d5-81c4-9d6559e49c2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:46.479953 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1a59c-c1bd-42d5-81c4-9d6559e49c2c None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:46.490773 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 637/1270] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:46.496439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc31cce-9b52-4f80-8914-249720ca70bb None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:46.528821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:46.600285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc31cce-9b52-4f80-8914-249720ca70bb None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:46.639854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ec20c-2e73-4c65-ac5b-d9139c9c37d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:46.640939 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ec20c-2e73-4c65-ac5b-d9139c9c37d9 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:46.641342 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ec20c-2e73-4c65-ac5b-d9139c9c37d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:46.641833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ec20c-2e73-4c65-ac5b-d9139c9c37d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:46.651039 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ec20c-2e73-4c65-ac5b-d9139c9c37d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:46.652992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ec20c-2e73-4c65-ac5b-d9139c9c37d9 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:46.669376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-427ec20c-2e73-4c65-ac5b-d9139c9c37d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:35:46.669376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:46.669376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:46.669376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:46.669376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:46.669376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:46.669376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:46.669376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:46.669376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:46.669376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:46.669376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:46.669376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:46.669376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:35:46.669376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:35:46.669376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:35:46.670460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:35:46.670460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:46.670460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:46.670460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:46.670460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:46.670460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:46.670460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:46.670460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:46.670460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:46.670460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:46.670460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:46.670460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:46.670460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:46.670460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:46.670460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:46.670460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:46.670460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:46.671736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:35:46.671736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:35:46.671736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:35:46.671736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:35:46.671736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:35:46.671736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:35:46.671736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:35:46.671736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:35:46.671736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 634/1271] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:35:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:46.676473 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1a59c-c1bd-42d5-81c4-9d6559e49c2c None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:46.734005 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:46.774482 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1a59c-c1bd-42d5-81c4-9d6559e49c2c None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:46.807506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdf7f0-13dc-4c52-a06f-a634b1e04454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:46.808163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdf7f0-13dc-4c52-a06f-a634b1e04454 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:46.808451 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdf7f0-13dc-4c52-a06f-a634b1e04454 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:46.808731 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdf7f0-13dc-4c52-a06f-a634b1e04454 None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:46.809513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdf7f0-13dc-4c52-a06f-a634b1e04454 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:46.809822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdf7f0-13dc-4c52-a06f-a634b1e04454 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:46.811256 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdf7f0-13dc-4c52-a06f-a634b1e04454 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:46.823115 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 638/1272] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:35:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:35:46.829078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ec20c-2e73-4c65-ac5b-d9139c9c37d9 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:46.916918 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:46.926690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ec20c-2e73-4c65-ac5b-d9139c9c37d9 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:46.960010 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e39e62-0cc0-4611-8c23-62950e40d087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:46.961256 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e39e62-0cc0-4611-8c23-62950e40d087 None admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:46.961538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e39e62-0cc0-4611-8c23-62950e40d087 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:46.961840 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e39e62-0cc0-4611-8c23-62950e40d087 None admin] PATH_INFO: `/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/93630ed5e6dc4a5f958bc7cf1a68a26c/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:46.964094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e39e62-0cc0-4611-8c23-62950e40d087 None admin] RBAC: Authorizing `identity:create_grant(project_id=47d2bb094c9843c9a7726f1d672d2433, user_id=93630ed5e6dc4a5f958bc7cf1a68a26c, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:46.966047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e39e62-0cc0-4611-8c23-62950e40d087 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:46.978444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 635/1273] 10.222.0.36 () {58 vars in 1439 bytes} [Wed Jun 22 11:35:46 2022] PUT /identity/v3/projects/47d2bb094c9843c9a7726f1d672d2433/users/93630ed5e6dc4a5f958bc7cf1a68a26c/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:35:48.987289 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56f808-893a-4b13-8321-9db6d768df02 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:48.987824 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56f808-893a-4b13-8321-9db6d768df02 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:48.987910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56f808-893a-4b13-8321-9db6d768df02 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:48.989215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 639/1274] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:35:48.994286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b7f58-a7ed-444b-b179-b68293502719 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:48.994724 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b7f58-a7ed-444b-b179-b68293502719 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:48.994825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b7f58-a7ed-444b-b179-b68293502719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:49.012445 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-b51b7f58-a7ed-444b-b179-b68293502719 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:35:49.013887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b51b7f58-a7ed-444b-b179-b68293502719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:35:49.089682 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 636/1275] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:35:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:35:49.113147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56f808-893a-4b13-8321-9db6d768df02 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:49.210940 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56f808-893a-4b13-8321-9db6d768df02 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:49.227513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:49.244119 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895fdcc-bcbd-4183-be1a-f3b0df370f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:49.244910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895fdcc-bcbd-4183-be1a-f3b0df370f72 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:49.245193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895fdcc-bcbd-4183-be1a-f3b0df370f72 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:49.245982 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895fdcc-bcbd-4183-be1a-f3b0df370f72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:49.255737 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c895fdcc-bcbd-4183-be1a-f3b0df370f72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 22 11:35:49.264156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-c895fdcc-bcbd-4183-be1a-f3b0df370f72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:35:49.264156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:49.264156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:49.264156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:49.264156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:49.264156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:49.264156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:49.264156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:49.264156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:49.264156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:49.264156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:49.264156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:49.264156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 22 11:35:49.264156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 22 11:35:49.264156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 22 11:35:49.264156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 22 11:35:49.265260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 22 11:35:49.265260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 22 11:35:49.265260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 22 11:35:49.265260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 22 11:35:49.265260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:49.265260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:49.265260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:49.265260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:49.265260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:49.265260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:49.265260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:49.265260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:49.265260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:49.265260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:49.265260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:49.265260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:49.266441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:49.266441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:49.266441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:49.266441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:49.266441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 22 11:35:49.266441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 22 11:35:49.266441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 22 11:35:49.266441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 22 11:35:49.266441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 22 11:35:49.266441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 22 11:35:49.266441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 22 11:35:49.266441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:49.266441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 640/1276] 10.222.0.36 () {56 vars in 1096 bytes} [Wed Jun 22 11:35:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:49.271400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b7f58-a7ed-444b-b179-b68293502719 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:49.348706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:49.367428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b7f58-a7ed-444b-b179-b68293502719 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:49.400133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457c564-4944-47d3-91ee-c91dad4cfff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:49.400794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457c564-4944-47d3-91ee-c91dad4cfff4 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:49.401076 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457c564-4944-47d3-91ee-c91dad4cfff4 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:49.401353 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457c564-4944-47d3-91ee-c91dad4cfff4 None admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:49.402098 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457c564-4944-47d3-91ee-c91dad4cfff4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:49.402404 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457c564-4944-47d3-91ee-c91dad4cfff4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:49.403791 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457c564-4944-47d3-91ee-c91dad4cfff4 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:49.411737 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 637/1277] 10.222.0.36 () {56 vars in 1099 bytes} [Wed Jun 22 11:35:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:49.418148 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895fdcc-bcbd-4183-be1a-f3b0df370f72 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:49.514004 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895fdcc-bcbd-4183-be1a-f3b0df370f72 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:49.550030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce60ef2-ec40-4b4f-be59-5ac763abec7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:49.550812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce60ef2-ec40-4b4f-be59-5ac763abec7c None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:49.551093 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce60ef2-ec40-4b4f-be59-5ac763abec7c None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:49.551386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce60ef2-ec40-4b4f-be59-5ac763abec7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:49.562311 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce60ef2-ec40-4b4f-be59-5ac763abec7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:49.563787 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce60ef2-ec40-4b4f-be59-5ac763abec7c None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:49.580235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-3ce60ef2-ec40-4b4f-be59-5ac763abec7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:49.580235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:49.580235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:49.580235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:49.580235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:49.580235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:49.580235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:49.580235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:49.580235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:49.580235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:49.580235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:49.580235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:49.580235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:35:49.580235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:35:49.580235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:35:49.581279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:35:49.581279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:49.581279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:49.581279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:49.581279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:49.581279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:49.581279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:49.581279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:49.581279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:49.581279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:49.581279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:49.581279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:49.581279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:49.581279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:49.581279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:49.581279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:49.581279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:49.582296 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:35:49.582296 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:35:49.582296 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:49.582296 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:49.582296 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 641/1278] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:49.587418 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457c564-4944-47d3-91ee-c91dad4cfff4 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:49.694411 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457c564-4944-47d3-91ee-c91dad4cfff4 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:49.726831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d4d27-6ef2-48e7-a97f-dea7c41dd5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:49.727577 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d4d27-6ef2-48e7-a97f-dea7c41dd5e9 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:49.727805 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d4d27-6ef2-48e7-a97f-dea7c41dd5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:49.728083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d4d27-6ef2-48e7-a97f-dea7c41dd5e9 None admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:49.728794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d4d27-6ef2-48e7-a97f-dea7c41dd5e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:49.729099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d4d27-6ef2-48e7-a97f-dea7c41dd5e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:49.731057 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d4d27-6ef2-48e7-a97f-dea7c41dd5e9 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:49.742478 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 638/1279] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:49.748157 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce60ef2-ec40-4b4f-be59-5ac763abec7c None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:49.778877 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:49.851967 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce60ef2-ec40-4b4f-be59-5ac763abec7c None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:49.884398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a68d83-1cbe-437d-9fe9-8d784cdc3c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:49.885185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a68d83-1cbe-437d-9fe9-8d784cdc3c4b None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:49.885464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a68d83-1cbe-437d-9fe9-8d784cdc3c4b None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:49.885738 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a68d83-1cbe-437d-9fe9-8d784cdc3c4b None admin] PATH_INFO: `/v3/users/placement` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:49.894019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a68d83-1cbe-437d-9fe9-8d784cdc3c4b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:49.895479 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a68d83-1cbe-437d-9fe9-8d784cdc3c4b None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:49.909337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-b3a68d83-1cbe-437d-9fe9-8d784cdc3c4b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 22 11:35:49.909337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:49.909337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:49.909337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:49.909337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:49.909337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:49.909337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:49.909337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:49.909337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:49.909337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:49.909337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:49.909337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:49.909337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 22 11:35:49.909337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 22 11:35:49.909337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 22 11:35:49.909337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 22 11:35:49.910496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:49.910496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:49.910496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 22 11:35:49.910496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:35:49.910496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 22 11:35:49.910496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 22 11:35:49.910496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:49.910496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:49.910496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:49.910496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:49.910496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:49.910496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:49.910496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:49.910496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:49.910496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:49.910496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:49.910496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:49.910496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:49.911856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:49.911856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:49.911856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 22 11:35:49.911856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 22 11:35:49.911856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 22 11:35:49.911856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 22 11:35:49.911856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 22 11:35:49.911856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 22 11:35:49.911856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 22 11:35:49.911856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:49.911856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 642/1280] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:49.916364 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d4d27-6ef2-48e7-a97f-dea7c41dd5e9 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:49.920173 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:50.017237 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d4d27-6ef2-48e7-a97f-dea7c41dd5e9 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:50.049337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7344b6-b41d-4a9e-bce1-3dbea274fdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:50.049927 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7344b6-b41d-4a9e-bce1-3dbea274fdcf None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:50.050212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7344b6-b41d-4a9e-bce1-3dbea274fdcf None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:50.050490 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7344b6-b41d-4a9e-bce1-3dbea274fdcf None admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:50.051312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7344b6-b41d-4a9e-bce1-3dbea274fdcf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:50.051648 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7344b6-b41d-4a9e-bce1-3dbea274fdcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:50.053054 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7344b6-b41d-4a9e-bce1-3dbea274fdcf None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:50.070189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 639/1281] 10.222.0.36 () {56 vars in 1143 bytes} [Wed Jun 22 11:35:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:50.076928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a68d83-1cbe-437d-9fe9-8d784cdc3c4b None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:50.165717 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:50.176065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a68d83-1cbe-437d-9fe9-8d784cdc3c4b None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:50.212799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40fc5c-d0c3-40e7-9d9e-70abd2e2c33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:50.213442 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40fc5c-d0c3-40e7-9d9e-70abd2e2c33a None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:50.213726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40fc5c-d0c3-40e7-9d9e-70abd2e2c33a None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:50.214003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40fc5c-d0c3-40e7-9d9e-70abd2e2c33a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:50.279516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40fc5c-d0c3-40e7-9d9e-70abd2e2c33a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:50.280971 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40fc5c-d0c3-40e7-9d9e-70abd2e2c33a None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:50.366032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:50.376778 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 643/1282] 10.222.0.36 () {58 vars in 1282 bytes} [Wed Jun 22 11:35:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 22 11:35:50.382708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7344b6-b41d-4a9e-bce1-3dbea274fdcf None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:50.442855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:50.480811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7344b6-b41d-4a9e-bce1-3dbea274fdcf None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:50.513554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa834c-c90f-447c-ae90-b16547d8ec34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:50.514341 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa834c-c90f-447c-ae90-b16547d8ec34 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:50.514625 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa834c-c90f-447c-ae90-b16547d8ec34 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:50.514901 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa834c-c90f-447c-ae90-b16547d8ec34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:50.525038 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa834c-c90f-447c-ae90-b16547d8ec34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:50.526465 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa834c-c90f-447c-ae90-b16547d8ec34 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:50.544497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-91fa834c-c90f-447c-ae90-b16547d8ec34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:50.544497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:50.544497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:50.544497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:50.544497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:50.544497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:50.544497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:50.544497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:50.544497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:50.544497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:50.544497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:50.544497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:50.544497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 22 11:35:50.544497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 22 11:35:50.544497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 22 11:35:50.545609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 22 11:35:50.545609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:50.545609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:50.545609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:50.545609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:50.545609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:50.545609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:50.545609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:50.545609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:50.545609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:50.545609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:50.545609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:50.545609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:50.545609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:50.545609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:50.545609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:50.545609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:50.546550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 22 11:35:50.546550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 22 11:35:50.546550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 22 11:35:50.546550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:35:50.546550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 640/1283] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:35:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:50.551841 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40fc5c-d0c3-40e7-9d9e-70abd2e2c33a None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:50.660456 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40fc5c-d0c3-40e7-9d9e-70abd2e2c33a None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:50.693755 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30732f50-ad69-4162-bdf7-d5281007a31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:50.694411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30732f50-ad69-4162-bdf7-d5281007a31d None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:50.694695 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30732f50-ad69-4162-bdf7-d5281007a31d None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:50.694969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30732f50-ad69-4162-bdf7-d5281007a31d None admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:50.695703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30732f50-ad69-4162-bdf7-d5281007a31d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:50.696010 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30732f50-ad69-4162-bdf7-d5281007a31d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:50.697382 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30732f50-ad69-4162-bdf7-d5281007a31d None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:50.708302 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 644/1284] 10.222.0.36 () {56 vars in 1109 bytes} [Wed Jun 22 11:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:50.714690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa834c-c90f-447c-ae90-b16547d8ec34 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:50.809194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa834c-c90f-447c-ae90-b16547d8ec34 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:50.822394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:50.842550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae223a2-534e-4656-aebd-bfcfa537b197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:50.843324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae223a2-534e-4656-aebd-bfcfa537b197 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:50.843638 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae223a2-534e-4656-aebd-bfcfa537b197 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:50.843911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae223a2-534e-4656-aebd-bfcfa537b197 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:50.850362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae223a2-534e-4656-aebd-bfcfa537b197 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:50.851824 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae223a2-534e-4656-aebd-bfcfa537b197 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:50.865605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application [None req-6ae223a2-534e-4656-aebd-bfcfa537b197 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:35:50.865605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:50.865605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:50.865605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:50.865605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:50.865605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:50.865605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:50.865605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:50.865605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:50.865605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:50.865605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:50.865605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:50.865605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 22 11:35:50.865605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 22 11:35:50.865605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 22 11:35:50.866687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 22 11:35:50.866687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:50.866687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:50.866687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:50.866687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:50.866687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:50.866687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:50.866687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:50.866687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:50.866687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:50.866687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:50.866687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:50.866687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:50.866687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:50.866687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:50.866687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:50.866687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:50.867834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 22 11:35:50.867834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 22 11:35:50.867834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 22 11:35:50.867834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 22 11:35:50.867834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 22 11:35:50.867834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 22 11:35:50.867834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 22 11:35:50.867834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: ERROR keystone.server.flask.application Jun 22 11:35:50.867834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 641/1285] 10.222.0.36 () {56 vars in 1111 bytes} [Wed Jun 22 11:35:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:50.872579 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30732f50-ad69-4162-bdf7-d5281007a31d None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:50.922844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:50.974948 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30732f50-ad69-4162-bdf7-d5281007a31d None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:51.007169 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f701e0-6788-4b03-8d08-176d4fd8cdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:51.007845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f701e0-6788-4b03-8d08-176d4fd8cdca None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:51.008130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f701e0-6788-4b03-8d08-176d4fd8cdca None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:51.008405 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f701e0-6788-4b03-8d08-176d4fd8cdca None admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:51.009182 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f701e0-6788-4b03-8d08-176d4fd8cdca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:51.009489 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f701e0-6788-4b03-8d08-176d4fd8cdca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:51.011441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f701e0-6788-4b03-8d08-176d4fd8cdca None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:51.023200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 645/1286] 10.222.0.36 () {56 vars in 1148 bytes} [Wed Jun 22 11:35:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:51.029947 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae223a2-534e-4656-aebd-bfcfa537b197 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:51.131766 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae223a2-534e-4656-aebd-bfcfa537b197 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:51.165435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d086af8-dcfb-45c6-b805-e6aaeed4917d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:51.166113 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d086af8-dcfb-45c6-b805-e6aaeed4917d None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:51.166394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d086af8-dcfb-45c6-b805-e6aaeed4917d None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:51.166670 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d086af8-dcfb-45c6-b805-e6aaeed4917d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:51.167550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d086af8-dcfb-45c6-b805-e6aaeed4917d None admin] RBAC: Adding query filter params (role.id=96c584428d924fdba9671949f760c406, scope.project.id=47d2bb094c9843c9a7726f1d672d2433, user.id=93630ed5e6dc4a5f958bc7cf1a68a26c) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:51.167864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d086af8-dcfb-45c6-b805-e6aaeed4917d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:51.169291 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d086af8-dcfb-45c6-b805-e6aaeed4917d None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:51.175738 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 642/1287] 10.222.0.36 () {56 vars in 1374 bytes} [Wed Jun 22 11:35:51 2022] GET /identity/v3/role_assignments?user.id=93630ed5e6dc4a5f958bc7cf1a68a26c&scope.project.id=47d2bb094c9843c9a7726f1d672d2433&role.id=96c584428d924fdba9671949f760c406 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:53.218430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff5d93-f575-4076-a24d-fc364850a592 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:53.218944 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff5d93-f575-4076-a24d-fc364850a592 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:53.218944 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff5d93-f575-4076-a24d-fc364850a592 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:53.220285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 646/1288] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:35:53.225258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d2d10-6625-476b-aeeb-6aa2b3747fa2 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:53.225671 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d2d10-6625-476b-aeeb-6aa2b3747fa2 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:53.225859 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d2d10-6625-476b-aeeb-6aa2b3747fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:53.242345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-328d2d10-6625-476b-aeeb-6aa2b3747fa2 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:35:53.243582 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-328d2d10-6625-476b-aeeb-6aa2b3747fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:35:53.277279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:53.314067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 643/1289] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:35:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:35:53.338637 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff5d93-f575-4076-a24d-fc364850a592 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:53.439363 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff5d93-f575-4076-a24d-fc364850a592 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:53.450227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:53.474086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78795ed2-bb1c-4457-b9cd-781342b8f0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:53.474811 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78795ed2-bb1c-4457-b9cd-781342b8f0dc None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:53.475091 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78795ed2-bb1c-4457-b9cd-781342b8f0dc None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:53.475393 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78795ed2-bb1c-4457-b9cd-781342b8f0dc None admin] PATH_INFO: `/v3/services/placement` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:53.476035 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78795ed2-bb1c-4457-b9cd-781342b8f0dc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:53.477384 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78795ed2-bb1c-4457-b9cd-781342b8f0dc None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:53.488856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-78795ed2-bb1c-4457-b9cd-781342b8f0dc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 22 11:35:53.488856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:53.488856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:35:53.488856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:35:53.488856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:35:53.488856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:35:53.488856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:35:53.488856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:35:53.488856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 22 11:35:53.488856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:53.488856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:35:53.488856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:53.488856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:53.488856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:53.488856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:53.488856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:53.488856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:53.490113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:53.490113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:53.490113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:53.490113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:53.490113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:53.490113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:53.490113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:35:53.490113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:35:53.490113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:35:53.490113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:35:53.490113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:53.490113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:53.490113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:53.490113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:53.490113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:53.490113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:53.491193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:53.491193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:53.491193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:53.491193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:53.491193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:53.491193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:53.491193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:53.491193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:53.491193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:53.491193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:53.491193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:35:53.491193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:35:53.491193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 22 11:35:53.491193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:53.491193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 647/1290] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jun 22 11:35:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:53.496427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d2d10-6625-476b-aeeb-6aa2b3747fa2 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:53.595497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d2d10-6625-476b-aeeb-6aa2b3747fa2 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:53.632296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73481b-a1f9-4d8b-b5c8-8fa092d80d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:53.632977 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73481b-a1f9-4d8b-b5c8-8fa092d80d1f None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:53.633263 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73481b-a1f9-4d8b-b5c8-8fa092d80d1f None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:53.633539 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73481b-a1f9-4d8b-b5c8-8fa092d80d1f None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:53.634245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73481b-a1f9-4d8b-b5c8-8fa092d80d1f None admin] RBAC: Adding query filter params (name=placement) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:53.634550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73481b-a1f9-4d8b-b5c8-8fa092d80d1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:53.636497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73481b-a1f9-4d8b-b5c8-8fa092d80d1f None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:53.642244 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 644/1291] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Jun 22 11:35:53 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:53.648545 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78795ed2-bb1c-4457-b9cd-781342b8f0dc None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:53.758762 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78795ed2-bb1c-4457-b9cd-781342b8f0dc None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:53.791506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec5012-2c05-4d9b-80df-ff808d18aecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:53.792159 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec5012-2c05-4d9b-80df-ff808d18aecd None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:53.792441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec5012-2c05-4d9b-80df-ff808d18aecd None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:53.792742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec5012-2c05-4d9b-80df-ff808d18aecd None admin] PATH_INFO: `/v3/services` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:53.793436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec5012-2c05-4d9b-80df-ff808d18aecd None admin] RBAC: Adding query filter params (type=placement) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:53.793764 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec5012-2c05-4d9b-80df-ff808d18aecd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:53.795134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec5012-2c05-4d9b-80df-ff808d18aecd None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:53.799514 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 648/1292] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Jun 22 11:35:53 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:55.810844 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be64db-6502-4382-840b-860fbd7da41c None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:55.811502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be64db-6502-4382-840b-860fbd7da41c None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:55.811502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be64db-6502-4382-840b-860fbd7da41c None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:55.812799 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 645/1293] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:35:55.817966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318872ad-cdf7-4596-a5a2-49dcd6ac0ea7 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:55.818343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318872ad-cdf7-4596-a5a2-49dcd6ac0ea7 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:55.818525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318872ad-cdf7-4596-a5a2-49dcd6ac0ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:55.835423 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-318872ad-cdf7-4596-a5a2-49dcd6ac0ea7 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:35:55.836613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-318872ad-cdf7-4596-a5a2-49dcd6ac0ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:35:55.864442 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:55.903631 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 649/1294] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:35:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:35:55.925869 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be64db-6502-4382-840b-860fbd7da41c None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:55.956990 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:56.028070 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be64db-6502-4382-840b-860fbd7da41c None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:56.063644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db50bb-b275-4403-b1e2-38376daf4140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:56.064225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db50bb-b275-4403-b1e2-38376daf4140 None admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:56.064506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db50bb-b275-4403-b1e2-38376daf4140 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:56.064785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db50bb-b275-4403-b1e2-38376daf4140 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:56.065513 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db50bb-b275-4403-b1e2-38376daf4140 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:56.066869 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db50bb-b275-4403-b1e2-38376daf4140 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:56.079936 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 646/1295] 10.222.0.36 () {60 vars in 1141 bytes} [Wed Jun 22 11:35:55 2022] POST /identity/v3/services => generated 246 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:35:58.118011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ca954-939e-4a69-9259-15017310fca7 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:58.118530 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ca954-939e-4a69-9259-15017310fca7 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:58.118623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ca954-939e-4a69-9259-15017310fca7 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:58.119959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 650/1296] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:35:58.125900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ff84e-2753-425d-867d-8cc00a410d90 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:58.126285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ff84e-2753-425d-867d-8cc00a410d90 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:58.126462 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ff84e-2753-425d-867d-8cc00a410d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:58.143514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-419ff84e-2753-425d-867d-8cc00a410d90 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:35:58.144708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-419ff84e-2753-425d-867d-8cc00a410d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:35:58.214574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 647/1297] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:35:58 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 22 11:35:58.237408 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ca954-939e-4a69-9259-15017310fca7 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:58.337343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ca954-939e-4a69-9259-15017310fca7 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:58.372347 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d96b13-f9ab-4f59-8542-158b5f8c6b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:58.373147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d96b13-f9ab-4f59-8542-158b5f8c6b07 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:58.373433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d96b13-f9ab-4f59-8542-158b5f8c6b07 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:58.373713 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d96b13-f9ab-4f59-8542-158b5f8c6b07 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:58.374366 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d96b13-f9ab-4f59-8542-158b5f8c6b07 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:58.375779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d96b13-f9ab-4f59-8542-158b5f8c6b07 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:58.388261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-22d96b13-f9ab-4f59-8542-158b5f8c6b07 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 22 11:35:58.388261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:58.388261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:35:58.388261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:35:58.388261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:35:58.388261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:35:58.388261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:35:58.388261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:35:58.388261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 22 11:35:58.388261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:58.388261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:35:58.388261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:58.388261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:35:58.388261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:35:58.388261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:35:58.388261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:35:58.388261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:35:58.389398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:35:58.389398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:35:58.389398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:35:58.389398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:35:58.389398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:35:58.389398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:35:58.389398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:35:58.389398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:35:58.389398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:35:58.389398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:35:58.389398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:35:58.389398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:35:58.389398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:35:58.389398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:35:58.389398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:35:58.389398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:35:58.390371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:35:58.390371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:35:58.390371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:35:58.390371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:35:58.390371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:35:58.390371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:35:58.390371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:35:58.390371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:35:58.390371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:35:58.390371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:35:58.390371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:35:58.390371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:35:58.390371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 22 11:35:58.390371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:35:58.390371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 651/1298] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jun 22 11:35:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:35:58.395499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ff84e-2753-425d-867d-8cc00a410d90 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:58.414198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:58.493623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ff84e-2753-425d-867d-8cc00a410d90 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:58.526777 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ec45c-a818-43ea-9e19-1396dd179dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:58.527464 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ec45c-a818-43ea-9e19-1396dd179dc6 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:58.527752 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ec45c-a818-43ea-9e19-1396dd179dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:58.528032 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ec45c-a818-43ea-9e19-1396dd179dc6 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:58.528744 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ec45c-a818-43ea-9e19-1396dd179dc6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:58.529052 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ec45c-a818-43ea-9e19-1396dd179dc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:58.530431 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ec45c-a818-43ea-9e19-1396dd179dc6 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:58.535728 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 648/1299] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Jun 22 11:35:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:58.541631 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d96b13-f9ab-4f59-8542-158b5f8c6b07 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:58.573282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:58.646367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d96b13-f9ab-4f59-8542-158b5f8c6b07 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:58.691232 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2c80d-90c1-4f87-9e6a-271acadea767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:58.691936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2c80d-90c1-4f87-9e6a-271acadea767 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:58.692226 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2c80d-90c1-4f87-9e6a-271acadea767 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:58.692505 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2c80d-90c1-4f87-9e6a-271acadea767 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:58.693430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2c80d-90c1-4f87-9e6a-271acadea767 None admin] RBAC: Adding query filter params (interface=public, service_id=d391f65ab7484f208c896978f5c9420c, region_id=RegionOne) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:35:58.693618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2c80d-90c1-4f87-9e6a-271acadea767 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:58.694998 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde2c80d-90c1-4f87-9e6a-271acadea767 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:58.704407 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 652/1300] 10.222.0.36 () {56 vars in 1251 bytes} [Wed Jun 22 11:35:58 2022] GET /identity/v3/endpoints?service_id=d391f65ab7484f208c896978f5c9420c&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:35:58.706695 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ec45c-a818-43ea-9e19-1396dd179dc6 None admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:35:58.786736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:35:58.804976 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ec45c-a818-43ea-9e19-1396dd179dc6 None admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:35:58.837289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e21a7c-9384-4072-8978-ae4d65ddf31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:35:58.837940 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e21a7c-9384-4072-8978-ae4d65ddf31d None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:35:58.838220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e21a7c-9384-4072-8978-ae4d65ddf31d None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:35:58.838491 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e21a7c-9384-4072-8978-ae4d65ddf31d None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:35:58.839755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e21a7c-9384-4072-8978-ae4d65ddf31d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:35:58.841106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e21a7c-9384-4072-8978-ae4d65ddf31d None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:35:58.847746 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 649/1301] 10.222.0.36 () {56 vars in 1087 bytes} [Wed Jun 22 11:35:58 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:36:00.862007 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e307634-6384-42ea-a593-82c0cbbc92e9 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:36:00.862554 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e307634-6384-42ea-a593-82c0cbbc92e9 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:36:00.862554 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e307634-6384-42ea-a593-82c0cbbc92e9 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:36:00.863947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 653/1302] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:36:00.868924 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fa40a-b108-4682-b028-bb850361ef7c None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:36:00.869326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fa40a-b108-4682-b028-bb850361ef7c None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:36:00.869463 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fa40a-b108-4682-b028-bb850361ef7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:36:00.886415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-eb5fa40a-b108-4682-b028-bb850361ef7c None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:36:00.887876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-eb5fa40a-b108-4682-b028-bb850361ef7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:36:00.955622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 650/1303] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:36:00 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 22 11:36:00.982279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e307634-6384-42ea-a593-82c0cbbc92e9 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:36:01.076334 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:36:01.083686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e307634-6384-42ea-a593-82c0cbbc92e9 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:36:01.117219 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75848b6d-0c6e-468a-8b34-e0ef053e60d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:36:01.117946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75848b6d-0c6e-468a-8b34-e0ef053e60d8 None admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:36:01.118236 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75848b6d-0c6e-468a-8b34-e0ef053e60d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:36:01.118513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75848b6d-0c6e-468a-8b34-e0ef053e60d8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:36:01.119163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75848b6d-0c6e-468a-8b34-e0ef053e60d8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:36:01.120591 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75848b6d-0c6e-468a-8b34-e0ef053e60d8 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:36:01.132241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application [None req-75848b6d-0c6e-468a-8b34-e0ef053e60d8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 22 11:36:01.132241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:36:01.132241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 22 11:36:01.132241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 22 11:36:01.132241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 22 11:36:01.132241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 22 11:36:01.132241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 22 11:36:01.132241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:36:01.132241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 22 11:36:01.132241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:36:01.132241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 22 11:36:01.132241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:36:01.132241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 22 11:36:01.132241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 22 11:36:01.132241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 22 11:36:01.132241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 22 11:36:01.132241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 22 11:36:01.133370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 22 11:36:01.133370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 22 11:36:01.133370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 22 11:36:01.133370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 22 11:36:01.133370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 22 11:36:01.133370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 22 11:36:01.133370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 22 11:36:01.133370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 22 11:36:01.133370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 22 11:36:01.133370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 22 11:36:01.133370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 22 11:36:01.133370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 22 11:36:01.133370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 22 11:36:01.133370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 22 11:36:01.133370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 22 11:36:01.133370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.get_or_create( Jun 22 11:36:01.134476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 22 11:36:01.134476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application with Lock( Jun 22 11:36:01.134476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 22 11:36:01.134476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self._enter() Jun 22 11:36:01.134476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 22 11:36:01.134476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 22 11:36:01.134476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 22 11:36:01.134476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application return self.creator() Jun 22 11:36:01.134476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 22 11:36:01.134476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application created_value = creator( Jun 22 11:36:01.134476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 22 11:36:01.134476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 22 11:36:01.134476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 22 11:36:01.134476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: ERROR keystone.server.flask.application Jun 22 11:36:01.134476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 654/1304] 10.222.0.36 () {56 vars in 1117 bytes} [Wed Jun 22 11:36:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 22 11:36:01.139753 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fa40a-b108-4682-b028-bb850361ef7c None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:36:01.230784 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:36:01.254590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fa40a-b108-4682-b028-bb850361ef7c None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:36:01.288348 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330be87-b3c7-4363-b336-99aa4de73695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:36:01.289021 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330be87-b3c7-4363-b336-99aa4de73695 None admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:36:01.289302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330be87-b3c7-4363-b336-99aa4de73695 None admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:36:01.289579 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330be87-b3c7-4363-b336-99aa4de73695 None admin] PATH_INFO: `/v3/services` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:36:01.290282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7330be87-b3c7-4363-b336-99aa4de73695 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:36:01.290591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7330be87-b3c7-4363-b336-99aa4de73695 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:36:01.292018 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7330be87-b3c7-4363-b336-99aa4de73695 None admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:36:01.297087 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 651/1305] 10.222.0.36 () {56 vars in 1116 bytes} [Wed Jun 22 11:36:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:36:01.302944 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75848b6d-0c6e-468a-8b34-e0ef053e60d8 None admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:36:01.408710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75848b6d-0c6e-468a-8b34-e0ef053e60d8 None admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:36:01.442721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae5a70-232b-4776-a62c-604907519e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:36:01.443394 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae5a70-232b-4776-a62c-604907519e30 None admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:36:01.443681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae5a70-232b-4776-a62c-604907519e30 None admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:36:01.443957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae5a70-232b-4776-a62c-604907519e30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:36:01.444698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae5a70-232b-4776-a62c-604907519e30 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:36:01.446072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae5a70-232b-4776-a62c-604907519e30 None admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:36:01.464345 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 655/1306] 10.222.0.36 () {60 vars in 1144 bytes} [Wed Jun 22 11:36:01 2022] POST /identity/v3/endpoints => generated 331 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:36:11.294408 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=30 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:36:11.460230 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:04.734258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb2152-e1a5-48f4-b2e8-31d31df531f0 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:04.734818 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb2152-e1a5-48f4-b2e8-31d31df531f0 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:04.734818 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb2152-e1a5-48f4-b2e8-31d31df531f0 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:04.741603 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5a586-1dee-4a50-bcec-42d6fb059abf None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:04.742620 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5a586-1dee-4a50-bcec-42d6fb059abf None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:04.742703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5a586-1dee-4a50-bcec-42d6fb059abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:04.745951 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 652/1307] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:37:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:37:04.822309 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-e4d5a586-1dee-4a50-bcec-42d6fb059abf None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:37:04.823604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e4d5a586-1dee-4a50-bcec-42d6fb059abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:04.948879 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 656/1308] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:37:05.004953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abe762-d6c6-4268-a4f6-b7ece3c7b636 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:05.005487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abe762-d6c6-4268-a4f6-b7ece3c7b636 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:05.005923 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abe762-d6c6-4268-a4f6-b7ece3c7b636 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:05.008746 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 653/1309] 10.222.0.36 () {54 vars in 964 bytes} [Wed Jun 22 11:37:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:37:05.015677 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cf2ac-242f-40d1-aaed-6a036b8e36b3 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:05.016292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cf2ac-242f-40d1-aaed-6a036b8e36b3 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:05.016673 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cf2ac-242f-40d1-aaed-6a036b8e36b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:05.127092 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-ec5cf2ac-242f-40d1-aaed-6a036b8e36b3 None None] MFA Rules not processed for user `c00d0bb777a044a3a4d4540d187d938a`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:37:05.129102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ec5cf2ac-242f-40d1-aaed-6a036b8e36b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:05.159980 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:05.225415 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 657/1310] 10.222.0.36 () {58 vars in 1061 bytes} [Wed Jun 22 11:37:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 22 11:37:05.234935 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abe762-d6c6-4268-a4f6-b7ece3c7b636 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:37:05.236847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f6abe762-d6c6-4268-a4f6-b7ece3c7b636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:05.355810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abe762-d6c6-4268-a4f6-b7ece3c7b636 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:37:05.430781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cededc52-1362-46c2-b3a0-a2f0559c03c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:37:05.431487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cededc52-1362-46c2-b3a0-a2f0559c03c9 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:05.431775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cededc52-1362-46c2-b3a0-a2f0559c03c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:05.432053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cededc52-1362-46c2-b3a0-a2f0559c03c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:05.433728 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-cededc52-1362-46c2-b3a0-a2f0559c03c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:05.488936 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:05.549011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cededc52-1362-46c2-b3a0-a2f0559c03c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:37:05.550452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cededc52-1362-46c2-b3a0-a2f0559c03c9 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:37:05.570063 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 654/1311] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:37:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:37:07.940914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59515755-4f14-4b4d-a15a-a72f457cd015 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:07.941431 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59515755-4f14-4b4d-a15a-a72f457cd015 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:07.941532 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59515755-4f14-4b4d-a15a-a72f457cd015 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:07.943108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 658/1312] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:37:07.950844 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb3fc7-628d-411c-b8b4-1ee9d8b7ea6f None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:07.952415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb3fc7-628d-411c-b8b4-1ee9d8b7ea6f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:07.952415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb3fc7-628d-411c-b8b4-1ee9d8b7ea6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:08.035044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-a1fb3fc7-628d-411c-b8b4-1ee9d8b7ea6f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:37:08.036282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a1fb3fc7-628d-411c-b8b4-1ee9d8b7ea6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:08.137387 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 655/1313] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:37:08.176290 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59515755-4f14-4b4d-a15a-a72f457cd015 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:37:08.177916 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-59515755-4f14-4b4d-a15a-a72f457cd015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:08.297742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59515755-4f14-4b4d-a15a-a72f457cd015 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:37:08.382920 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:08.385679 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c53a39-4996-4fd9-9248-4f3de14b9283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:37:08.386846 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c53a39-4996-4fd9-9248-4f3de14b9283 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:08.387367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c53a39-4996-4fd9-9248-4f3de14b9283 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:08.387680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c53a39-4996-4fd9-9248-4f3de14b9283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:08.389950 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-20c53a39-4996-4fd9-9248-4f3de14b9283 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:08.506776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c53a39-4996-4fd9-9248-4f3de14b9283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:37:08.508285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c53a39-4996-4fd9-9248-4f3de14b9283 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:37:08.527393 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 659/1314] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:37:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:37:10.905871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9aac3f-79d1-4064-b4bb-3596ee42d589 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:10.906401 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9aac3f-79d1-4064-b4bb-3596ee42d589 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:10.906492 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9aac3f-79d1-4064-b4bb-3596ee42d589 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:10.907845 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 656/1315] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:37:10.913089 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae390246-6d84-4fed-a082-9aa0e9d6f0e5 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:10.913555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae390246-6d84-4fed-a082-9aa0e9d6f0e5 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:10.913858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae390246-6d84-4fed-a082-9aa0e9d6f0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:10.985919 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-ae390246-6d84-4fed-a082-9aa0e9d6f0e5 None None] MFA Rules not processed for user `d6d9a2f332cf40a6b5bc0d534f1eab4b`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:37:10.987155 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ae390246-6d84-4fed-a082-9aa0e9d6f0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:11.041022 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:11.087988 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 660/1316] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:37:10 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:37:11.141695 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11a1e4-cd84-4b40-8f27-ca108d019dd8 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:11.142077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11a1e4-cd84-4b40-8f27-ca108d019dd8 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:11.142430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11a1e4-cd84-4b40-8f27-ca108d019dd8 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:11.143933 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 657/1317] 10.222.0.36 () {54 vars in 964 bytes} [Wed Jun 22 11:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:37:11.149994 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddab860-797a-4259-bb88-df9eed139381 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:11.150358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddab860-797a-4259-bb88-df9eed139381 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:11.150708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddab860-797a-4259-bb88-df9eed139381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:11.233819 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-7ddab860-797a-4259-bb88-df9eed139381 None None] MFA Rules not processed for user `c00d0bb777a044a3a4d4540d187d938a`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:37:11.235028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-7ddab860-797a-4259-bb88-df9eed139381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:11.312185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 661/1318] 10.222.0.36 () {58 vars in 1061 bytes} [Wed Jun 22 11:37:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:37:11.321355 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11a1e4-cd84-4b40-8f27-ca108d019dd8 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:37:11.322874 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2f11a1e4-cd84-4b40-8f27-ca108d019dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:11.332446 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:11.426986 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11a1e4-cd84-4b40-8f27-ca108d019dd8 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:37:11.505648 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b94d8-58bd-4fe9-8679-a73db52076f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:37:11.506290 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b94d8-58bd-4fe9-8679-a73db52076f3 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:11.506568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b94d8-58bd-4fe9-8679-a73db52076f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:11.506841 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b94d8-58bd-4fe9-8679-a73db52076f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:11.509085 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-766b94d8-58bd-4fe9-8679-a73db52076f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:11.570561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:11.615355 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b94d8-58bd-4fe9-8679-a73db52076f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:37:11.617320 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b94d8-58bd-4fe9-8679-a73db52076f3 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:37:11.637108 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 658/1319] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:37:11 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:37:14.611242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e57f6-d542-4dbe-8212-643553997df4 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:14.611809 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e57f6-d542-4dbe-8212-643553997df4 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:14.611912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e57f6-d542-4dbe-8212-643553997df4 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:14.613204 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 662/1320] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:37:14.618946 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47297a7-6a85-4547-9d8e-b5ccfe7d9178 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:14.619535 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47297a7-6a85-4547-9d8e-b5ccfe7d9178 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:14.619535 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47297a7-6a85-4547-9d8e-b5ccfe7d9178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:14.684201 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-b47297a7-6a85-4547-9d8e-b5ccfe7d9178 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:37:14.685417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b47297a7-6a85-4547-9d8e-b5ccfe7d9178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:14.778726 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 659/1321] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:37:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:37:14.818139 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e57f6-d542-4dbe-8212-643553997df4 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:37:14.821368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-4a6e57f6-d542-4dbe-8212-643553997df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:14.878684 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:14.934827 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e57f6-d542-4dbe-8212-643553997df4 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:37:15.032562 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5cf3e-228a-49ba-a4c6-a167ff283aa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:37:15.033209 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5cf3e-228a-49ba-a4c6-a167ff283aa9 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:15.033494 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5cf3e-228a-49ba-a4c6-a167ff283aa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:15.033772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5cf3e-228a-49ba-a4c6-a167ff283aa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:15.035447 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b5e5cf3e-228a-49ba-a4c6-a167ff283aa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:15.134302 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:15.148957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e5cf3e-228a-49ba-a4c6-a167ff283aa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:37:15.150400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e5cf3e-228a-49ba-a4c6-a167ff283aa9 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:37:15.171050 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 663/1322] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:37:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:37:17.967710 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490efec3-11b4-4626-b9e5-8e01ff0459d4 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:17.968261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490efec3-11b4-4626-b9e5-8e01ff0459d4 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:17.968378 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490efec3-11b4-4626-b9e5-8e01ff0459d4 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:17.969687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 660/1323] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:37:17.975211 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df973d-2b34-4500-b6bd-60813fa1ab98 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:17.976190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df973d-2b34-4500-b6bd-60813fa1ab98 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:17.976363 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df973d-2b34-4500-b6bd-60813fa1ab98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:18.047172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-44df973d-2b34-4500-b6bd-60813fa1ab98 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:37:18.048431 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-44df973d-2b34-4500-b6bd-60813fa1ab98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:18.153488 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 664/1324] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:37:18.191322 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490efec3-11b4-4626-b9e5-8e01ff0459d4 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:37:18.193083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-490efec3-11b4-4626-b9e5-8e01ff0459d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:18.215567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:18.305410 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490efec3-11b4-4626-b9e5-8e01ff0459d4 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:37:18.379041 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fbb83-a332-4e22-8e0c-d590b11cb629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:37:18.379738 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fbb83-a332-4e22-8e0c-d590b11cb629 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:18.380020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fbb83-a332-4e22-8e0c-d590b11cb629 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:18.380296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fbb83-a332-4e22-8e0c-d590b11cb629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:18.381935 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b22fbb83-a332-4e22-8e0c-d590b11cb629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:18.453475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:18.492135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fbb83-a332-4e22-8e0c-d590b11cb629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:37:18.493486 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fbb83-a332-4e22-8e0c-d590b11cb629 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:37:18.512373 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 661/1325] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:37:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:37:20.627903 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbbf34-2c17-4f5b-8203-02c6af9519ba None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:20.628452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbbf34-2c17-4f5b-8203-02c6af9519ba None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:20.628452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbbf34-2c17-4f5b-8203-02c6af9519ba None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:20.629948 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 665/1326] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:37:20.635669 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca48973-371c-4070-a146-4b3480e63eb7 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:20.636109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca48973-371c-4070-a146-4b3480e63eb7 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:20.636476 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca48973-371c-4070-a146-4b3480e63eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:20.702244 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-fca48973-371c-4070-a146-4b3480e63eb7 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:37:20.703494 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-fca48973-371c-4070-a146-4b3480e63eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:20.801061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 662/1327] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:37:20.837353 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbbf34-2c17-4f5b-8203-02c6af9519ba None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:37:20.838930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-1bdbbf34-2c17-4f5b-8203-02c6af9519ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:20.902827 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:20.948796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbbf34-2c17-4f5b-8203-02c6af9519ba None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:37:21.028750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c3237-dd52-4ee7-9842-dfb5819440a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:37:21.029400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c3237-dd52-4ee7-9842-dfb5819440a7 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:21.029681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c3237-dd52-4ee7-9842-dfb5819440a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:21.029955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c3237-dd52-4ee7-9842-dfb5819440a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:21.031628 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-591c3237-dd52-4ee7-9842-dfb5819440a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:21.138900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:21.141188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c3237-dd52-4ee7-9842-dfb5819440a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:37:21.143409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c3237-dd52-4ee7-9842-dfb5819440a7 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:37:21.162113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 666/1328] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:37:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:37:23.314392 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60320d62-adf8-41c0-a378-f223a070bbe3 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:23.314924 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60320d62-adf8-41c0-a378-f223a070bbe3 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:23.314924 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60320d62-adf8-41c0-a378-f223a070bbe3 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:23.316550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 663/1329] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:37:23.322618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf5d0a-6199-4211-9459-d40b77995d03 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:23.323094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf5d0a-6199-4211-9459-d40b77995d03 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:23.323199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf5d0a-6199-4211-9459-d40b77995d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:23.400636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-b0cf5d0a-6199-4211-9459-d40b77995d03 None None] MFA Rules not processed for user `d6d9a2f332cf40a6b5bc0d534f1eab4b`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:37:23.401846 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b0cf5d0a-6199-4211-9459-d40b77995d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:23.504418 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 667/1330] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:37:23 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:37:23.542678 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60320d62-adf8-41c0-a378-f223a070bbe3 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:37:23.544307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-60320d62-adf8-41c0-a378-f223a070bbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:23.578825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:23.657893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60320d62-adf8-41c0-a378-f223a070bbe3 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:37:23.735507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73917bb-64d7-4721-b2c6-9b23292ecd6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:37:23.736217 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73917bb-64d7-4721-b2c6-9b23292ecd6b service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:23.736571 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73917bb-64d7-4721-b2c6-9b23292ecd6b service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:23.736907 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73917bb-64d7-4721-b2c6-9b23292ecd6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:23.738590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c73917bb-64d7-4721-b2c6-9b23292ecd6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:23.825965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:23.875660 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73917bb-64d7-4721-b2c6-9b23292ecd6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:37:23.877125 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73917bb-64d7-4721-b2c6-9b23292ecd6b service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:37:23.897398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 664/1331] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:37:23 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:37:26.248959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a2f14-d5f4-4df8-938c-7b78b1c45935 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:26.248959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a2f14-d5f4-4df8-938c-7b78b1c45935 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:26.248959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a2f14-d5f4-4df8-938c-7b78b1c45935 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:26.249909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 668/1332] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:37:26.258505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08a99d-9bd4-46e3-9ab2-3ef47439b338 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:26.259260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08a99d-9bd4-46e3-9ab2-3ef47439b338 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:26.259260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08a99d-9bd4-46e3-9ab2-3ef47439b338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:26.329748 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-de08a99d-9bd4-46e3-9ab2-3ef47439b338 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:37:26.331021 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-de08a99d-9bd4-46e3-9ab2-3ef47439b338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:26.436240 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 665/1333] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:37:26.473639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a2f14-d5f4-4df8-938c-7b78b1c45935 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:37:26.475227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-310a2f14-d5f4-4df8-938c-7b78b1c45935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:26.554481 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:26.602643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a2f14-d5f4-4df8-938c-7b78b1c45935 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:37:26.696403 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b4a8b-15da-4517-99aa-ff42af399076 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:37:26.697061 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b4a8b-15da-4517-99aa-ff42af399076 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:26.697343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b4a8b-15da-4517-99aa-ff42af399076 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:26.697619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b4a8b-15da-4517-99aa-ff42af399076 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:26.699341 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-983b4a8b-15da-4517-99aa-ff42af399076 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:26.827393 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b4a8b-15da-4517-99aa-ff42af399076 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:37:26.828867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b4a8b-15da-4517-99aa-ff42af399076 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:37:26.845310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:26.858568 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 669/1334] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:37:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:37:29.045277 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a503df1-83dc-4ac5-ab70-20568b129c75 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:29.045806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a503df1-83dc-4ac5-ab70-20568b129c75 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:29.046231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a503df1-83dc-4ac5-ab70-20568b129c75 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:29.048759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 666/1335] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:37:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:37:29.056486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dd5bb-0fe2-40c3-bd27-1452eac8e268 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:29.057273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dd5bb-0fe2-40c3-bd27-1452eac8e268 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:29.057432 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dd5bb-0fe2-40c3-bd27-1452eac8e268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:29.128604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-507dd5bb-0fe2-40c3-bd27-1452eac8e268 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:37:29.129841 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-507dd5bb-0fe2-40c3-bd27-1452eac8e268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:29.255130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 670/1336] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:37:29.295715 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a503df1-83dc-4ac5-ab70-20568b129c75 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:37:29.297252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-8a503df1-83dc-4ac5-ab70-20568b129c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:29.343538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:29.413969 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a503df1-83dc-4ac5-ab70-20568b129c75 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:37:29.494061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df23a7-f951-414d-b011-2d8a71e3ea4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:37:29.494712 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df23a7-f951-414d-b011-2d8a71e3ea4c service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:29.494991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df23a7-f951-414d-b011-2d8a71e3ea4c service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:29.495267 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df23a7-f951-414d-b011-2d8a71e3ea4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:29.496958 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a3df23a7-f951-414d-b011-2d8a71e3ea4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:29.591890 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:29.622778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df23a7-f951-414d-b011-2d8a71e3ea4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:37:29.624262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df23a7-f951-414d-b011-2d8a71e3ea4c service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:37:29.650874 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 667/1337] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:37:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:37:32.467134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81b4d5-195e-4ea4-8633-37bb07910af3 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:32.467846 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81b4d5-195e-4ea4-8633-37bb07910af3 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:32.467846 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81b4d5-195e-4ea4-8633-37bb07910af3 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:32.469134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 671/1338] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:37:32.476387 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d34d7-d712-48b2-9dc6-c9f810637c9f None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:32.476908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d34d7-d712-48b2-9dc6-c9f810637c9f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:32.477001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d34d7-d712-48b2-9dc6-c9f810637c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:32.581074 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-d39d34d7-d712-48b2-9dc6-c9f810637c9f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:37:32.582335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d39d34d7-d712-48b2-9dc6-c9f810637c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:32.699072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 668/1339] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:37:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:37:32.735278 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81b4d5-195e-4ea4-8633-37bb07910af3 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:37:32.736870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-4e81b4d5-195e-4ea4-8633-37bb07910af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:32.827854 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:32.859849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81b4d5-195e-4ea4-8633-37bb07910af3 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:37:32.940141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab880c0-3160-4d0a-9435-f6d5619beee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:37:32.940816 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab880c0-3160-4d0a-9435-f6d5619beee8 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:32.941169 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab880c0-3160-4d0a-9435-f6d5619beee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:32.941366 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab880c0-3160-4d0a-9435-f6d5619beee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:32.943112 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5ab880c0-3160-4d0a-9435-f6d5619beee8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:33.064385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab880c0-3160-4d0a-9435-f6d5619beee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:37:33.065762 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab880c0-3160-4d0a-9435-f6d5619beee8 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:37:33.087028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 672/1340] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:37:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:37:37.276185 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5b045-6a04-44c1-baaf-382eec65ce57 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:37.276759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5b045-6a04-44c1-baaf-382eec65ce57 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:37.276759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5b045-6a04-44c1-baaf-382eec65ce57 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:37.278110 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 669/1341] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:37:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:37:37.284503 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee145d8-97d8-426c-9833-1c72dd168ac7 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:37.285001 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee145d8-97d8-426c-9833-1c72dd168ac7 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:37.285135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee145d8-97d8-426c-9833-1c72dd168ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:37.292656 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:37.379397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-7ee145d8-97d8-426c-9833-1c72dd168ac7 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:37:37.380607 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-7ee145d8-97d8-426c-9833-1c72dd168ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:37.488739 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 673/1342] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:37:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:37:37.528500 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5b045-6a04-44c1-baaf-382eec65ce57 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:37:37.530750 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-3cf5b045-6a04-44c1-baaf-382eec65ce57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:37.581324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:37.628685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5b045-6a04-44c1-baaf-382eec65ce57 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:37:37.633745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8af905-a90e-47d5-a981-55aad33c07c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:37:37.634512 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8af905-a90e-47d5-a981-55aad33c07c5 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:37.636221 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8af905-a90e-47d5-a981-55aad33c07c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:37.636221 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8af905-a90e-47d5-a981-55aad33c07c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:37.651165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8af905-a90e-47d5-a981-55aad33c07c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:37:37.652652 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8af905-a90e-47d5-a981-55aad33c07c5 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:37:37.665528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 670/1343] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:37:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:37:40.561033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803270f8-973d-458c-8443-eae738a180cc None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:40.561648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803270f8-973d-458c-8443-eae738a180cc None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:40.561648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803270f8-973d-458c-8443-eae738a180cc None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:40.563064 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 674/1344] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:37:40.569795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30310f6d-a5f2-46b7-84cb-7f6d46b66b54 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:40.570381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30310f6d-a5f2-46b7-84cb-7f6d46b66b54 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:40.570381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30310f6d-a5f2-46b7-84cb-7f6d46b66b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:40.601078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-30310f6d-a5f2-46b7-84cb-7f6d46b66b54 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:37:40.602308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-30310f6d-a5f2-46b7-84cb-7f6d46b66b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:40.703908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 671/1345] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:37:40 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:37:40.742194 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803270f8-973d-458c-8443-eae738a180cc None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:37:40.753107 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803270f8-973d-458c-8443-eae738a180cc None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:37:40.756751 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2aea6-e11c-41ee-81dc-280bc986c757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:37:40.757403 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2aea6-e11c-41ee-81dc-280bc986c757 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:40.757683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2aea6-e11c-41ee-81dc-280bc986c757 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:40.758052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2aea6-e11c-41ee-81dc-280bc986c757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:40.768320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2aea6-e11c-41ee-81dc-280bc986c757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:37:40.769759 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2aea6-e11c-41ee-81dc-280bc986c757 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:37:40.780183 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 675/1346] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:37:40 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:37:45.362502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36508a-afe1-4926-a618-8ef2a7062250 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:45.362502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36508a-afe1-4926-a618-8ef2a7062250 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:45.363522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36508a-afe1-4926-a618-8ef2a7062250 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:45.364044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 672/1347] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:37:45.370202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ab30b-0d70-43b0-a5ee-1ed412a55cf8 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:45.370998 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ab30b-0d70-43b0-a5ee-1ed412a55cf8 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:45.370998 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ab30b-0d70-43b0-a5ee-1ed412a55cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:45.390645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-160ab30b-0d70-43b0-a5ee-1ed412a55cf8 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:37:45.391914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-160ab30b-0d70-43b0-a5ee-1ed412a55cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:45.400665 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 676/1348] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:37:45.439141 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36508a-afe1-4926-a618-8ef2a7062250 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:37:45.450906 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36508a-afe1-4926-a618-8ef2a7062250 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:37:45.454582 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6b366a-b3f2-4e30-be25-ea440826d107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:37:45.455246 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b366a-b3f2-4e30-be25-ea440826d107 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:45.455561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b366a-b3f2-4e30-be25-ea440826d107 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:45.455842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6b366a-b3f2-4e30-be25-ea440826d107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:45.466482 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b366a-b3f2-4e30-be25-ea440826d107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:37:45.467793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6b366a-b3f2-4e30-be25-ea440826d107 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:37:45.480171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 673/1349] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:37:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:37:50.746869 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58179c-afa9-490d-b83a-f5607284c066 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:50.747520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58179c-afa9-490d-b83a-f5607284c066 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:50.747520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be58179c-afa9-490d-b83a-f5607284c066 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:50.750424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 677/1350] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jun 22 11:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:37:50.756042 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec09c9-95b4-4f72-afdf-506ca5348e3b None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:50.756667 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec09c9-95b4-4f72-afdf-506ca5348e3b None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:50.757510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec09c9-95b4-4f72-afdf-506ca5348e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:50.847995 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-baec09c9-95b4-4f72-afdf-506ca5348e3b None None] MFA Rules not processed for user `93630ed5e6dc4a5f958bc7cf1a68a26c`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:37:50.849938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-baec09c9-95b4-4f72-afdf-506ca5348e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:50.881338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:50.950133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 674/1351] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jun 22 11:37:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:37:55.384549 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=72 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:37:55.550593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b35cf-880c-4e4e-a510-3b4f9faa87fa None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:55.551140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b35cf-880c-4e4e-a510-3b4f9faa87fa None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:55.551634 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b35cf-880c-4e4e-a510-3b4f9faa87fa None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:55.554528 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 678/1352] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jun 22 11:37:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:37:55.560236 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe996b18-4331-465a-adee-a80453e11423 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:55.560741 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe996b18-4331-465a-adee-a80453e11423 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:55.561014 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe996b18-4331-465a-adee-a80453e11423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:55.581612 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-fe996b18-4331-465a-adee-a80453e11423 None None] MFA Rules not processed for user `93630ed5e6dc4a5f958bc7cf1a68a26c`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:37:55.582834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-fe996b18-4331-465a-adee-a80453e11423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:55.595497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 675/1353] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jun 22 11:37:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:37:57.291188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28143d3-bf37-44c3-a106-21851f88bb03 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:57.291772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28143d3-bf37-44c3-a106-21851f88bb03 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:57.291772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28143d3-bf37-44c3-a106-21851f88bb03 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:57.293049 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 679/1354] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jun 22 11:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:37:57.298247 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e649cd-f8f0-402e-836c-0c6ba7f86052 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:37:57.298710 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e649cd-f8f0-402e-836c-0c6ba7f86052 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:37:57.298947 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e649cd-f8f0-402e-836c-0c6ba7f86052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:37:57.317237 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-13e649cd-f8f0-402e-836c-0c6ba7f86052 None None] MFA Rules not processed for user `93630ed5e6dc4a5f958bc7cf1a68a26c`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:37:57.318921 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-13e649cd-f8f0-402e-836c-0c6ba7f86052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:37:57.328258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 676/1355] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jun 22 11:37:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:38:00.957075 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec28610-7f68-4c12-a085-52215a8c1710 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:00.960072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec28610-7f68-4c12-a085-52215a8c1710 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:00.960687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec28610-7f68-4c12-a085-52215a8c1710 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:00.964117 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 680/1356] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:38:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:00.971057 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976b24f-8fb4-4990-b655-1536aeeb5e42 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:00.974455 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976b24f-8fb4-4990-b655-1536aeeb5e42 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:00.975119 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976b24f-8fb4-4990-b655-1536aeeb5e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:01.025011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-1976b24f-8fb4-4990-b655-1536aeeb5e42 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:01.026407 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-1976b24f-8fb4-4990-b655-1536aeeb5e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:01.038587 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 677/1357] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:01.101005 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf88898-d18c-428f-aa80-1e39584b2007 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:01.101509 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf88898-d18c-428f-aa80-1e39584b2007 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:01.101981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf88898-d18c-428f-aa80-1e39584b2007 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:01.103701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 681/1358] 10.222.0.36 () {54 vars in 961 bytes} [Wed Jun 22 11:38:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:01.119040 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626dba0c-7028-4919-b30f-bbc970b3a3a6 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:01.119814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626dba0c-7028-4919-b30f-bbc970b3a3a6 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:01.121024 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626dba0c-7028-4919-b30f-bbc970b3a3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:01.197356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-626dba0c-7028-4919-b30f-bbc970b3a3a6 None None] MFA Rules not processed for user `479a376dcac14c509c1c55f0046d88cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:01.198895 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-626dba0c-7028-4919-b30f-bbc970b3a3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:01.322570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 678/1359] 10.222.0.36 () {58 vars in 1058 bytes} [Wed Jun 22 11:38:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:01.323458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf88898-d18c-428f-aa80-1e39584b2007 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:01.344963 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf88898-d18c-428f-aa80-1e39584b2007 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:01.354141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7bc48-bf43-4685-8efe-3d688d8d8d89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:01.356579 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7bc48-bf43-4685-8efe-3d688d8d8d89 service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:01.357421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7bc48-bf43-4685-8efe-3d688d8d8d89 service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:01.358300 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7bc48-bf43-4685-8efe-3d688d8d8d89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:01.376264 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7bc48-bf43-4685-8efe-3d688d8d8d89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:01.378177 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7bc48-bf43-4685-8efe-3d688d8d8d89 service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:01.390596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 682/1360] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:03.680723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f4d05-2a15-4755-99fc-953ab5a82590 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:03.681286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f4d05-2a15-4755-99fc-953ab5a82590 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:03.681286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f4d05-2a15-4755-99fc-953ab5a82590 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:03.682790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 679/1361] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:03.688062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a418df0-6092-43f1-919f-1d7d38fba1dc None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:03.688507 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a418df0-6092-43f1-919f-1d7d38fba1dc None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:03.688595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a418df0-6092-43f1-919f-1d7d38fba1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:03.706873 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-7a418df0-6092-43f1-919f-1d7d38fba1dc None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:03.708261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-7a418df0-6092-43f1-919f-1d7d38fba1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:03.716297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 683/1362] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:38:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:38:03.760139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f4d05-2a15-4755-99fc-953ab5a82590 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:03.775568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f4d05-2a15-4755-99fc-953ab5a82590 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:03.782089 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff655ddd-5cf4-439f-9b39-22b8e97911d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:03.783015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff655ddd-5cf4-439f-9b39-22b8e97911d8 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:03.783543 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff655ddd-5cf4-439f-9b39-22b8e97911d8 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:03.784074 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff655ddd-5cf4-439f-9b39-22b8e97911d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:03.794427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff655ddd-5cf4-439f-9b39-22b8e97911d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:03.796612 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff655ddd-5cf4-439f-9b39-22b8e97911d8 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:03.808341 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 680/1363] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:06.205004 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49534c93-c0c5-4296-8e27-93e905479616 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:06.205829 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49534c93-c0c5-4296-8e27-93e905479616 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:06.206261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49534c93-c0c5-4296-8e27-93e905479616 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:06.208204 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 684/1364] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:38:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:06.213582 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf70801-79c6-4077-9837-46e76cb5a7b2 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:06.214093 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf70801-79c6-4077-9837-46e76cb5a7b2 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:06.214232 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf70801-79c6-4077-9837-46e76cb5a7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:06.241043 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-ebf70801-79c6-4077-9837-46e76cb5a7b2 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:06.242254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ebf70801-79c6-4077-9837-46e76cb5a7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:06.250740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 681/1365] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:38:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:06.297122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137bba90-99e7-4811-b65b-92381b8e0289 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:06.297389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137bba90-99e7-4811-b65b-92381b8e0289 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:06.297660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137bba90-99e7-4811-b65b-92381b8e0289 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:06.299054 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 685/1366] 10.222.0.36 () {54 vars in 961 bytes} [Wed Jun 22 11:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:06.305208 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480472e-6eb3-47b9-8297-df0b9eed2e7b None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:06.305497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480472e-6eb3-47b9-8297-df0b9eed2e7b None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:06.305775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480472e-6eb3-47b9-8297-df0b9eed2e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:06.324152 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-4480472e-6eb3-47b9-8297-df0b9eed2e7b None None] MFA Rules not processed for user `479a376dcac14c509c1c55f0046d88cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:06.325330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-4480472e-6eb3-47b9-8297-df0b9eed2e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:06.334742 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 682/1367] 10.222.0.36 () {58 vars in 1058 bytes} [Wed Jun 22 11:38:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:38:06.342490 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137bba90-99e7-4811-b65b-92381b8e0289 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:06.357633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137bba90-99e7-4811-b65b-92381b8e0289 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:06.362972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3ab0dc-52b4-451a-b37b-931a4ff8dff9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:06.363631 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ab0dc-52b4-451a-b37b-931a4ff8dff9 service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:06.363911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ab0dc-52b4-451a-b37b-931a4ff8dff9 service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:06.366148 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ab0dc-52b4-451a-b37b-931a4ff8dff9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:06.373359 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3ab0dc-52b4-451a-b37b-931a4ff8dff9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:06.374936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3ab0dc-52b4-451a-b37b-931a4ff8dff9 service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:06.384265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 686/1368] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:08.541933 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0075c-8543-4b0f-a281-6fae68f27c95 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:08.542506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0075c-8543-4b0f-a281-6fae68f27c95 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:08.542506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0075c-8543-4b0f-a281-6fae68f27c95 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:08.543942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 683/1369] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:08.549693 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22e523-6579-4f7d-9010-a684490b5ba0 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:08.550128 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22e523-6579-4f7d-9010-a684490b5ba0 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:08.550229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22e523-6579-4f7d-9010-a684490b5ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:08.569252 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-6e22e523-6579-4f7d-9010-a684490b5ba0 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:08.570813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-6e22e523-6579-4f7d-9010-a684490b5ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:08.579727 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 687/1370] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:08.623917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0075c-8543-4b0f-a281-6fae68f27c95 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:08.642869 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0075c-8543-4b0f-a281-6fae68f27c95 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:08.649144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b6bde-a6a7-4455-a503-def5244e5f0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:08.649795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b6bde-a6a7-4455-a503-def5244e5f0c service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:08.650078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b6bde-a6a7-4455-a503-def5244e5f0c service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:08.650356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b6bde-a6a7-4455-a503-def5244e5f0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:08.659689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b6bde-a6a7-4455-a503-def5244e5f0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:08.661285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b6bde-a6a7-4455-a503-def5244e5f0c service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:08.672871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 684/1371] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:09.222032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a937a-c4d0-4db2-a578-c2f11bce9665 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:09.222567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a937a-c4d0-4db2-a578-c2f11bce9665 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:09.222997 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a937a-c4d0-4db2-a578-c2f11bce9665 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:09.225379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 688/1372] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:38:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:09.230995 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf5d48-6fe4-44cd-8492-324b0c69d21e None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:09.231503 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf5d48-6fe4-44cd-8492-324b0c69d21e None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:09.231934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf5d48-6fe4-44cd-8492-324b0c69d21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:09.250144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c8cf5d48-6fe4-44cd-8492-324b0c69d21e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:09.251650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c8cf5d48-6fe4-44cd-8492-324b0c69d21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:09.267515 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 685/1373] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:38:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:09.604207 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2c089-3442-4579-8935-73fd1eabf2dd None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:09.604207 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2c089-3442-4579-8935-73fd1eabf2dd None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:09.604207 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2c089-3442-4579-8935-73fd1eabf2dd None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:09.607079 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 689/1374] 10.222.0.36 () {54 vars in 963 bytes} [Wed Jun 22 11:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:09.610402 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d2917-130f-436d-bd05-4f38e81fe619 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:09.611021 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d2917-130f-436d-bd05-4f38e81fe619 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:09.611021 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d2917-130f-436d-bd05-4f38e81fe619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:09.618081 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:38:09.664766 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-f43d2917-130f-436d-bd05-4f38e81fe619 None None] MFA Rules not processed for user `ceb170ac6353495b9d1dc6723d59567a`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:09.665964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f43d2917-130f-436d-bd05-4f38e81fe619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:09.731510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 686/1375] 10.222.0.36 () {58 vars in 1060 bytes} [Wed Jun 22 11:38:09 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:09.735887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2c089-3442-4579-8935-73fd1eabf2dd None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:09.747444 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2c089-3442-4579-8935-73fd1eabf2dd None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:09.751235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c7c47c-2936-49e9-b769-db9c16a61e47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:09.752109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7c47c-2936-49e9-b769-db9c16a61e47 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:09.752561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7c47c-2936-49e9-b769-db9c16a61e47 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:09.752999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c7c47c-2936-49e9-b769-db9c16a61e47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:09.763783 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7c47c-2936-49e9-b769-db9c16a61e47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:09.765389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c7c47c-2936-49e9-b769-db9c16a61e47 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:09.775308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 690/1376] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:10.798455 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d17d2-7d31-4761-93cd-c6210a66a08e None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:10.798979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d17d2-7d31-4761-93cd-c6210a66a08e None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:10.799109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d17d2-7d31-4761-93cd-c6210a66a08e None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:10.800602 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 687/1377] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:10.806073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b1050-a99f-4846-b9e2-d1413505e783 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:10.806745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b1050-a99f-4846-b9e2-d1413505e783 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:10.806875 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b1050-a99f-4846-b9e2-d1413505e783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:10.831547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-1c9b1050-a99f-4846-b9e2-d1413505e783 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:10.833863 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-1c9b1050-a99f-4846-b9e2-d1413505e783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:10.841904 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 691/1378] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:38:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:10.886104 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d17d2-7d31-4761-93cd-c6210a66a08e None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:10.902255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d17d2-7d31-4761-93cd-c6210a66a08e None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:10.907900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df1cf2-1c22-4637-b952-aa60fdaf9c4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:10.908537 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df1cf2-1c22-4637-b952-aa60fdaf9c4f service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:10.908820 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df1cf2-1c22-4637-b952-aa60fdaf9c4f service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:10.909095 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df1cf2-1c22-4637-b952-aa60fdaf9c4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:10.917457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df1cf2-1c22-4637-b952-aa60fdaf9c4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:10.919027 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df1cf2-1c22-4637-b952-aa60fdaf9c4f service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:10.928811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 688/1379] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:13.007169 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc0884-9984-4f37-a18e-aa6f4d051510 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:13.007739 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc0884-9984-4f37-a18e-aa6f4d051510 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:13.007739 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc0884-9984-4f37-a18e-aa6f4d051510 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:13.009135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 692/1380] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:13.014632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e5e6e-21a2-4f1f-8d46-b582fe8b6025 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:13.015072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e5e6e-21a2-4f1f-8d46-b582fe8b6025 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:13.015169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e5e6e-21a2-4f1f-8d46-b582fe8b6025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:13.038280 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-7d2e5e6e-21a2-4f1f-8d46-b582fe8b6025 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:13.039646 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-7d2e5e6e-21a2-4f1f-8d46-b582fe8b6025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:13.047615 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 689/1381] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:38:13.092253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc0884-9984-4f37-a18e-aa6f4d051510 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:13.106486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc0884-9984-4f37-a18e-aa6f4d051510 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:13.111955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fda8c6-c56b-4b0e-b577-8f467be4eff8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:13.112629 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fda8c6-c56b-4b0e-b577-8f467be4eff8 service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:13.112915 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fda8c6-c56b-4b0e-b577-8f467be4eff8 service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:13.113220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fda8c6-c56b-4b0e-b577-8f467be4eff8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:13.121373 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fda8c6-c56b-4b0e-b577-8f467be4eff8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:13.122948 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fda8c6-c56b-4b0e-b577-8f467be4eff8 service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:13.133137 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 693/1382] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:15.245647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566bd41-ef1c-4cbc-ad0b-5b2d9ec2abdb None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:15.246227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566bd41-ef1c-4cbc-ad0b-5b2d9ec2abdb None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:15.246227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566bd41-ef1c-4cbc-ad0b-5b2d9ec2abdb None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:15.247631 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 690/1383] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:15.253132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c9a99-638b-4f4e-9dd1-855c95058649 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:15.253603 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c9a99-638b-4f4e-9dd1-855c95058649 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:15.253705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c9a99-638b-4f4e-9dd1-855c95058649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:15.272797 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-111c9a99-638b-4f4e-9dd1-855c95058649 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:15.274015 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-111c9a99-638b-4f4e-9dd1-855c95058649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:15.282186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 694/1384] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:15.324909 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566bd41-ef1c-4cbc-ad0b-5b2d9ec2abdb None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:15.340067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566bd41-ef1c-4cbc-ad0b-5b2d9ec2abdb None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:15.345827 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10f8dc-f594-4ffb-8e2f-75eb88712d51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:15.346488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10f8dc-f594-4ffb-8e2f-75eb88712d51 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:15.346775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10f8dc-f594-4ffb-8e2f-75eb88712d51 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:15.347056 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10f8dc-f594-4ffb-8e2f-75eb88712d51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:15.357192 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10f8dc-f594-4ffb-8e2f-75eb88712d51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:15.359066 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10f8dc-f594-4ffb-8e2f-75eb88712d51 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:15.372012 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 691/1385] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:17.526976 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cd0d4-199f-4ada-99a0-5edfd6383a2f None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:17.527719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cd0d4-199f-4ada-99a0-5edfd6383a2f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:17.527719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cd0d4-199f-4ada-99a0-5edfd6383a2f None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:17.528885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 695/1386] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:17.535153 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c7a6d-2b75-423e-9665-4ec02318ee77 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:17.535687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c7a6d-2b75-423e-9665-4ec02318ee77 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:17.535783 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c7a6d-2b75-423e-9665-4ec02318ee77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:17.562038 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-348c7a6d-2b75-423e-9665-4ec02318ee77 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:17.563276 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-348c7a6d-2b75-423e-9665-4ec02318ee77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:17.572838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 692/1387] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:17.629699 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cd0d4-199f-4ada-99a0-5edfd6383a2f None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:17.647820 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cd0d4-199f-4ada-99a0-5edfd6383a2f None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:17.655187 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7946688-737a-44db-a8b9-c1941ee1f14c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:17.656223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7946688-737a-44db-a8b9-c1941ee1f14c service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:17.656638 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7946688-737a-44db-a8b9-c1941ee1f14c service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:17.657069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7946688-737a-44db-a8b9-c1941ee1f14c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:17.677488 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7946688-737a-44db-a8b9-c1941ee1f14c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:17.680091 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7946688-737a-44db-a8b9-c1941ee1f14c service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:17.700800 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 696/1388] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:19.813641 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8d5e2-53bd-491d-8410-22357292ee08 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:19.814232 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8d5e2-53bd-491d-8410-22357292ee08 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:19.814232 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8d5e2-53bd-491d-8410-22357292ee08 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:19.815607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 693/1389] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:19.821575 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c0397-c478-4520-b686-a2532fb6890d None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:19.822076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c0397-c478-4520-b686-a2532fb6890d None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:19.822188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c0397-c478-4520-b686-a2532fb6890d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:19.840971 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-7c2c0397-c478-4520-b686-a2532fb6890d None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:19.842185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-7c2c0397-c478-4520-b686-a2532fb6890d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:19.850278 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 697/1390] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:19.906483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8d5e2-53bd-491d-8410-22357292ee08 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:19.924400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8d5e2-53bd-491d-8410-22357292ee08 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:19.932103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c4143-eff9-42b5-a97a-6eb9b781b51b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:19.932944 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c4143-eff9-42b5-a97a-6eb9b781b51b service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:19.933317 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c4143-eff9-42b5-a97a-6eb9b781b51b service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:19.933671 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c4143-eff9-42b5-a97a-6eb9b781b51b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:19.945729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c4143-eff9-42b5-a97a-6eb9b781b51b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:19.948073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c4143-eff9-42b5-a97a-6eb9b781b51b service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:19.961725 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 694/1391] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:22.139802 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dd95e-6f76-4cc5-9538-77dd9bd7393a None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:22.140651 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dd95e-6f76-4cc5-9538-77dd9bd7393a None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:22.140651 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dd95e-6f76-4cc5-9538-77dd9bd7393a None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:22.141821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 698/1392] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:22.147657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b400183-4b00-42a1-807b-18805253f948 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:22.148139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b400183-4b00-42a1-807b-18805253f948 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:22.148383 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b400183-4b00-42a1-807b-18805253f948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:22.170782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-1b400183-4b00-42a1-807b-18805253f948 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:22.172411 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-1b400183-4b00-42a1-807b-18805253f948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:22.181866 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 695/1393] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:22.228371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dd95e-6f76-4cc5-9538-77dd9bd7393a None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:22.242645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dd95e-6f76-4cc5-9538-77dd9bd7393a None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:22.248846 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed5235-3880-4d89-8a78-5f985c01eab0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:22.249721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed5235-3880-4d89-8a78-5f985c01eab0 service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:22.250082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed5235-3880-4d89-8a78-5f985c01eab0 service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:22.250458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed5235-3880-4d89-8a78-5f985c01eab0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:22.262841 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed5235-3880-4d89-8a78-5f985c01eab0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:22.265173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed5235-3880-4d89-8a78-5f985c01eab0 service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:22.278538 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 699/1394] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:24.493278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7f0a7-30f4-487b-a0b5-923f00e88257 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:24.493923 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7f0a7-30f4-487b-a0b5-923f00e88257 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:24.493923 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7f0a7-30f4-487b-a0b5-923f00e88257 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:24.495235 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 696/1395] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:24.500702 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b834f-5235-41ec-adcb-ce7f287301f6 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:24.501156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b834f-5235-41ec-adcb-ce7f287301f6 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:24.501414 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b834f-5235-41ec-adcb-ce7f287301f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:24.521974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d81b834f-5235-41ec-adcb-ce7f287301f6 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:24.523188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d81b834f-5235-41ec-adcb-ce7f287301f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:24.531227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 700/1396] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:24.578102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7f0a7-30f4-487b-a0b5-923f00e88257 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:24.593677 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7f0a7-30f4-487b-a0b5-923f00e88257 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:24.599137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4880d3-242b-4adf-9720-d3b4e263c7ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:24.599809 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4880d3-242b-4adf-9720-d3b4e263c7ad service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:24.600101 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4880d3-242b-4adf-9720-d3b4e263c7ad service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:24.600383 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4880d3-242b-4adf-9720-d3b4e263c7ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:24.608856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4880d3-242b-4adf-9720-d3b4e263c7ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:24.610446 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4880d3-242b-4adf-9720-d3b4e263c7ad service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:24.620901 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 697/1397] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:25.046515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42798b90-cc9f-4df1-ac2b-41a75413ffc6 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:25.046801 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42798b90-cc9f-4df1-ac2b-41a75413ffc6 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:25.047081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42798b90-cc9f-4df1-ac2b-41a75413ffc6 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:25.048535 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 701/1398] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:25.055433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c3e35-98be-4fbf-ab1b-4136b889f5d5 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:25.055798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c3e35-98be-4fbf-ab1b-4136b889f5d5 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:25.056148 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c3e35-98be-4fbf-ab1b-4136b889f5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:25.073732 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-e52c3e35-98be-4fbf-ab1b-4136b889f5d5 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:25.074928 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e52c3e35-98be-4fbf-ab1b-4136b889f5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:25.082831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 698/1399] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:38:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:25.139996 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b55ce-fad6-4462-b81a-5322e61747d1 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:25.140295 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b55ce-fad6-4462-b81a-5322e61747d1 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:25.140701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b55ce-fad6-4462-b81a-5322e61747d1 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:25.142107 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 702/1400] 10.222.0.36 () {54 vars in 962 bytes} [Wed Jun 22 11:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:25.147493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42a78d-b86b-4ef9-b765-bc33c6ae881f None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:25.147785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42a78d-b86b-4ef9-b765-bc33c6ae881f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:25.148065 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42a78d-b86b-4ef9-b765-bc33c6ae881f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:25.198292 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-7b42a78d-b86b-4ef9-b765-bc33c6ae881f None None] MFA Rules not processed for user `7ed133721ed044708ee362c93f564d07`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:25.199413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-7b42a78d-b86b-4ef9-b765-bc33c6ae881f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:25.232669 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:38:25.262089 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 699/1401] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Jun 22 11:38:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:25.269799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b55ce-fad6-4462-b81a-5322e61747d1 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:25.280621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b55ce-fad6-4462-b81a-5322e61747d1 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:25.284120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890cfbd-e6be-4c35-8e74-1e4be1c5f433 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ed133721ed044708ee362c93f564d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:25.284757 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890cfbd-e6be-4c35-8e74-1e4be1c5f433 service glance] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:25.285043 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890cfbd-e6be-4c35-8e74-1e4be1c5f433 service glance] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:25.285320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890cfbd-e6be-4c35-8e74-1e4be1c5f433 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:25.295019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890cfbd-e6be-4c35-8e74-1e4be1c5f433 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:25.296491 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890cfbd-e6be-4c35-8e74-1e4be1c5f433 service glance] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:25.306469 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 703/1402] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:26.030863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4644d-0b99-4b67-969e-df9c3bfee105 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:26.031338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4644d-0b99-4b67-969e-df9c3bfee105 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:26.031519 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4644d-0b99-4b67-969e-df9c3bfee105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:26.082231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-9fe4644d-0b99-4b67-969e-df9c3bfee105 None None] MFA Rules not processed for user `955c44ffe1bd426e900c7405df44d1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:26.084123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-9fe4644d-0b99-4b67-969e-df9c3bfee105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:26.148478 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 700/1403] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jun 22 11:38:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:26.189137 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e2723-6284-4b76-a457-041a64647d04 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:26.189624 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e2723-6284-4b76-a457-041a64647d04 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:26.189846 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e2723-6284-4b76-a457-041a64647d04 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:26.191340 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 704/1404] 10.222.0.36 () {54 vars in 942 bytes} [Wed Jun 22 11:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:26.198421 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0ee7c-d643-48f9-a095-9922e097bbfa None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:26.198713 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0ee7c-d643-48f9-a095-9922e097bbfa None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:26.199019 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0ee7c-d643-48f9-a095-9922e097bbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:26.252630 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-41a0ee7c-d643-48f9-a095-9922e097bbfa None None] MFA Rules not processed for user `dcd642e290e44e67b56eb3b8361d7af3`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:26.253838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-41a0ee7c-d643-48f9-a095-9922e097bbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:26.325992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 701/1405] 10.222.0.36 () {58 vars in 1039 bytes} [Wed Jun 22 11:38:26 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:26.335846 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e2723-6284-4b76-a457-041a64647d04 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:26.343437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=61 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:38:26.350704 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e2723-6284-4b76-a457-041a64647d04 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:26.355606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce2236-76b4-4730-8c5e-897321f67393 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd642e290e44e67b56eb3b8361d7af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:26.356249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce2236-76b4-4730-8c5e-897321f67393 service swift] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:26.356534 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce2236-76b4-4730-8c5e-897321f67393 service swift] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:26.356810 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce2236-76b4-4730-8c5e-897321f67393 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:26.365902 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce2236-76b4-4730-8c5e-897321f67393 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:26.367601 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce2236-76b4-4730-8c5e-897321f67393 service swift] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:26.376218 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 705/1406] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jun 22 11:38:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 22 11:38:26.728102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122797cf-57e3-4bf0-b4e3-b9eecf095d51 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:26.728300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122797cf-57e3-4bf0-b4e3-b9eecf095d51 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:26.728564 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122797cf-57e3-4bf0-b4e3-b9eecf095d51 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:26.729946 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 702/1407] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:26.735218 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0e2e2-4235-4b97-aa69-737526f598e5 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:26.735693 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0e2e2-4235-4b97-aa69-737526f598e5 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:26.736599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0e2e2-4235-4b97-aa69-737526f598e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:26.756322 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-95e0e2e2-4235-4b97-aa69-737526f598e5 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:26.757816 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-95e0e2e2-4235-4b97-aa69-737526f598e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:26.766618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 706/1408] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:26.810676 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122797cf-57e3-4bf0-b4e3-b9eecf095d51 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:26.829047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122797cf-57e3-4bf0-b4e3-b9eecf095d51 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:26.836354 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd286289-0a99-47c4-8864-87e35b10aa14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:26.837014 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd286289-0a99-47c4-8864-87e35b10aa14 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:26.837292 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd286289-0a99-47c4-8864-87e35b10aa14 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:26.837566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd286289-0a99-47c4-8864-87e35b10aa14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:26.846488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd286289-0a99-47c4-8864-87e35b10aa14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:26.848157 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd286289-0a99-47c4-8864-87e35b10aa14 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:26.856832 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:38:26.874063 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 703/1409] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:36.750183 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=8 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:38:49.017151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1261378-c663-491c-b115-dbfcd7438701 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:49.017815 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1261378-c663-491c-b115-dbfcd7438701 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:49.017815 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1261378-c663-491c-b115-dbfcd7438701 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:49.019060 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 707/1410] 10.222.0.36 () {54 vars in 918 bytes} [Wed Jun 22 11:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:49.024566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec9868-ab50-43a2-bb80-017bd21a4b32 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:49.025006 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec9868-ab50-43a2-bb80-017bd21a4b32 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:49.025117 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec9868-ab50-43a2-bb80-017bd21a4b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:49.043442 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c2ec9868-ab50-43a2-bb80-017bd21a4b32 None None] MFA Rules not processed for user `93630ed5e6dc4a5f958bc7cf1a68a26c`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:49.044671 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c2ec9868-ab50-43a2-bb80-017bd21a4b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:49.053705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 704/1411] 10.222.0.36 () {58 vars in 1015 bytes} [Wed Jun 22 11:38:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:49.068014 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baac0dd-fd7c-40d7-b6e0-aecacba00800 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:49.068299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baac0dd-fd7c-40d7-b6e0-aecacba00800 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:49.068577 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baac0dd-fd7c-40d7-b6e0-aecacba00800 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:49.069918 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 708/1412] 10.222.0.36 () {54 vars in 918 bytes} [Wed Jun 22 11:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:49.074479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b484c-558c-479d-84e5-d8a89b58c2ab None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:49.074785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b484c-558c-479d-84e5-d8a89b58c2ab None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:49.075071 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b484c-558c-479d-84e5-d8a89b58c2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:49.095902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-057b484c-558c-479d-84e5-d8a89b58c2ab None None] MFA Rules not processed for user `479a376dcac14c509c1c55f0046d88cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:49.097133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-057b484c-558c-479d-84e5-d8a89b58c2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:49.106939 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 705/1413] 10.222.0.36 () {58 vars in 1015 bytes} [Wed Jun 22 11:38:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:51.599736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a862c7-b3fa-4b91-a984-537b71dceaf3 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:51.600267 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a862c7-b3fa-4b91-a984-537b71dceaf3 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:51.600267 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a862c7-b3fa-4b91-a984-537b71dceaf3 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:51.601897 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 22 11:38:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:51.607304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11a5ba-ee49-40c2-abb5-c04b9a2158e7 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:51.607751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11a5ba-ee49-40c2-abb5-c04b9a2158e7 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:51.607912 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c11a5ba-ee49-40c2-abb5-c04b9a2158e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:51.625726 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-3c11a5ba-ee49-40c2-abb5-c04b9a2158e7 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:51.626909 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-3c11a5ba-ee49-40c2-abb5-c04b9a2158e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:51.634620 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 22 11:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:51.679657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a862c7-b3fa-4b91-a984-537b71dceaf3 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:51.694660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a862c7-b3fa-4b91-a984-537b71dceaf3 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:51.699910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba375c8-6a37-4b19-9951-bf7cafd5fc30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:51.700541 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba375c8-6a37-4b19-9951-bf7cafd5fc30 service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:51.700829 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba375c8-6a37-4b19-9951-bf7cafd5fc30 service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:51.701090 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba375c8-6a37-4b19-9951-bf7cafd5fc30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:51.709688 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba375c8-6a37-4b19-9951-bf7cafd5fc30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:51.711267 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba375c8-6a37-4b19-9951-bf7cafd5fc30 service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:51.720495 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 710/1416] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:52.615713 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd58a06-a9b1-4595-910a-65086eef2009 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:52.616253 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd58a06-a9b1-4595-910a-65086eef2009 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:52.616253 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd58a06-a9b1-4595-910a-65086eef2009 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:52.617644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 22 11:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:52.622736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70511b-c06b-495c-a810-0da58b85a8db None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:52.623122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70511b-c06b-495c-a810-0da58b85a8db None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:52.623341 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70511b-c06b-495c-a810-0da58b85a8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:52.641204 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-5e70511b-c06b-495c-a810-0da58b85a8db None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:52.642389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5e70511b-c06b-495c-a810-0da58b85a8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:52.650621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 22 11:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:52.804718 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd58a06-a9b1-4595-910a-65086eef2009 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:52.819568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd58a06-a9b1-4595-910a-65086eef2009 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:52.824896 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9104d-4034-4788-97fa-74666a4bb825 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:52.825532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9104d-4034-4788-97fa-74666a4bb825 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:52.825814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9104d-4034-4788-97fa-74666a4bb825 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:52.826088 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9104d-4034-4788-97fa-74666a4bb825 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:52.834728 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9104d-4034-4788-97fa-74666a4bb825 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:52.836356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9104d-4034-4788-97fa-74666a4bb825 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:52.845635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 708/1419] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:53.735269 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbc2d9-4089-44dc-9272-4ab38e96df46 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:53.735870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbc2d9-4089-44dc-9272-4ab38e96df46 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:53.735870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbc2d9-4089-44dc-9272-4ab38e96df46 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:53.737223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 22 11:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:53.742261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad4a4c-c888-46df-9744-44c2df4a876e None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:53.742687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad4a4c-c888-46df-9744-44c2df4a876e None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:53.742800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad4a4c-c888-46df-9744-44c2df4a876e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:53.761912 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-1bad4a4c-c888-46df-9744-44c2df4a876e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:53.763391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-1bad4a4c-c888-46df-9744-44c2df4a876e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:53.772202 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 22 11:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:53.922115 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbc2d9-4089-44dc-9272-4ab38e96df46 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:53.936936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbc2d9-4089-44dc-9272-4ab38e96df46 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:53.942341 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99dc3b7-c084-4a51-9f00-bfc09c6727db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:53.943601 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dc3b7-c084-4a51-9f00-bfc09c6727db service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:53.943930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dc3b7-c084-4a51-9f00-bfc09c6727db service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:53.944142 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dc3b7-c084-4a51-9f00-bfc09c6727db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:53.952694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99dc3b7-c084-4a51-9f00-bfc09c6727db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:53.954261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99dc3b7-c084-4a51-9f00-bfc09c6727db service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:53.963576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 713/1422] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:54.825457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe2b9a-608c-47c7-b981-4d909f50cc0e None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:54.826324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe2b9a-608c-47c7-b981-4d909f50cc0e None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:54.826602 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe2b9a-608c-47c7-b981-4d909f50cc0e None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:54.828023 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 22 11:38:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:54.833014 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634a2d4-e460-460d-953d-3cb7c920011d None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:54.833435 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634a2d4-e460-460d-953d-3cb7c920011d None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:54.833569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634a2d4-e460-460d-953d-3cb7c920011d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:54.851125 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-6634a2d4-e460-460d-953d-3cb7c920011d None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:54.852361 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-6634a2d4-e460-460d-953d-3cb7c920011d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:54.860048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 22 11:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:55.013629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe2b9a-608c-47c7-b981-4d909f50cc0e None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:55.027552 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe2b9a-608c-47c7-b981-4d909f50cc0e None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:55.032804 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf20298-90c5-4f32-bb5c-9ac5ea4f0e36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:55.033442 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf20298-90c5-4f32-bb5c-9ac5ea4f0e36 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:55.033721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf20298-90c5-4f32-bb5c-9ac5ea4f0e36 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:55.033992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf20298-90c5-4f32-bb5c-9ac5ea4f0e36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:55.042484 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf20298-90c5-4f32-bb5c-9ac5ea4f0e36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:55.044083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf20298-90c5-4f32-bb5c-9ac5ea4f0e36 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:55.053938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 711/1425] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:55.920162 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5ef88-2472-45c4-8db6-2ecdd7581ba3 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:55.920729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5ef88-2472-45c4-8db6-2ecdd7581ba3 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:55.920729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e5ef88-2472-45c4-8db6-2ecdd7581ba3 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:55.922085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 22 11:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:55.927018 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ae6f8-b9da-4dae-9882-65d5dad79d1f None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:55.927535 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ae6f8-b9da-4dae-9882-65d5dad79d1f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:55.931209 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ae6f8-b9da-4dae-9882-65d5dad79d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:55.948914 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-4a9ae6f8-b9da-4dae-9882-65d5dad79d1f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:55.950924 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-4a9ae6f8-b9da-4dae-9882-65d5dad79d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:55.961060 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 22 11:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:56.111827 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5ef88-2472-45c4-8db6-2ecdd7581ba3 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:56.126759 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e5ef88-2472-45c4-8db6-2ecdd7581ba3 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:56.132984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e8383-459f-403f-a63a-072e61b1d51d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:56.133622 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e8383-459f-403f-a63a-072e61b1d51d service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:56.133898 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e8383-459f-403f-a63a-072e61b1d51d service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:56.134168 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e8383-459f-403f-a63a-072e61b1d51d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:56.142796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e8383-459f-403f-a63a-072e61b1d51d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:56.144407 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e8383-459f-403f-a63a-072e61b1d51d service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:56.153637 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 716/1428] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:56.955179 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6a823-345e-4cd5-a963-4c9207bcd61e None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:56.955680 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6a823-345e-4cd5-a963-4c9207bcd61e None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:56.955782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6a823-345e-4cd5-a963-4c9207bcd61e None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:56.957140 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 22 11:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:56.961996 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be4b3d-4cc9-4bc9-beca-045fd69c08a3 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:56.962424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be4b3d-4cc9-4bc9-beca-045fd69c08a3 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:56.962572 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be4b3d-4cc9-4bc9-beca-045fd69c08a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:56.980018 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-67be4b3d-4cc9-4bc9-beca-045fd69c08a3 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:56.981199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-67be4b3d-4cc9-4bc9-beca-045fd69c08a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:56.988960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 22 11:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:57.138334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6a823-345e-4cd5-a963-4c9207bcd61e None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:57.152603 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6a823-345e-4cd5-a963-4c9207bcd61e None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:57.157921 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78954027-4ca4-4100-a0aa-04e37356ffa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:57.158552 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78954027-4ca4-4100-a0aa-04e37356ffa4 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:57.158831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78954027-4ca4-4100-a0aa-04e37356ffa4 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:57.159106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78954027-4ca4-4100-a0aa-04e37356ffa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:57.167302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78954027-4ca4-4100-a0aa-04e37356ffa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:57.168894 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78954027-4ca4-4100-a0aa-04e37356ffa4 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:57.178574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 714/1431] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:58.005146 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf6ce2-8a27-4154-85f1-583ef1a34e6a None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:58.005698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf6ce2-8a27-4154-85f1-583ef1a34e6a None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:58.005804 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf6ce2-8a27-4154-85f1-583ef1a34e6a None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:58.007112 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 22 11:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:58.012547 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e0a85-8002-4c5b-88f9-159f621a9975 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:58.013283 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e0a85-8002-4c5b-88f9-159f621a9975 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:58.013377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e0a85-8002-4c5b-88f9-159f621a9975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:58.032530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-5d0e0a85-8002-4c5b-88f9-159f621a9975 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:58.033746 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-5d0e0a85-8002-4c5b-88f9-159f621a9975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:58.043182 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 22 11:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:58.196027 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf6ce2-8a27-4154-85f1-583ef1a34e6a None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:58.210307 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf6ce2-8a27-4154-85f1-583ef1a34e6a None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:58.215612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad0ec6-b82c-493e-b58f-b39893d23b3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:58.216244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad0ec6-b82c-493e-b58f-b39893d23b3e service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:58.216523 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad0ec6-b82c-493e-b58f-b39893d23b3e service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:58.216796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad0ec6-b82c-493e-b58f-b39893d23b3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:58.225407 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad0ec6-b82c-493e-b58f-b39893d23b3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:58.226977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad0ec6-b82c-493e-b58f-b39893d23b3e service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:58.236166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 719/1434] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:38:59.046407 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a210e-176d-4285-a7c9-215ebf983648 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:59.046983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a210e-176d-4285-a7c9-215ebf983648 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:59.046983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a210e-176d-4285-a7c9-215ebf983648 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:59.048428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 22 11:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:38:59.054128 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ebba8-7ae9-4a4a-afde-22022bd005aa None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:59.054585 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ebba8-7ae9-4a4a-afde-22022bd005aa None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:59.054679 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ebba8-7ae9-4a4a-afde-22022bd005aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:59.072418 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d27ebba8-7ae9-4a4a-afde-22022bd005aa None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:38:59.073915 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d27ebba8-7ae9-4a4a-afde-22022bd005aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:38:59.081752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 22 11:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:38:59.242271 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a210e-176d-4285-a7c9-215ebf983648 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:38:59.256831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a210e-176d-4285-a7c9-215ebf983648 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:38:59.262218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05793f-338b-489b-a7e3-6d9c91aa8231 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:38:59.263110 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05793f-338b-489b-a7e3-6d9c91aa8231 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:38:59.263424 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05793f-338b-489b-a7e3-6d9c91aa8231 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:38:59.263707 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05793f-338b-489b-a7e3-6d9c91aa8231 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:38:59.272243 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a05793f-338b-489b-a7e3-6d9c91aa8231 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:38:59.273823 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a05793f-338b-489b-a7e3-6d9c91aa8231 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:38:59.283580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 717/1437] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:38:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:39:00.088017 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed60dec-ebc3-4641-8707-d0fe01b9b582 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:00.088597 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed60dec-ebc3-4641-8707-d0fe01b9b582 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:00.088597 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed60dec-ebc3-4641-8707-d0fe01b9b582 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:00.089986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 22 11:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:39:00.094887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefef3d3-f2a0-4608-81ab-65ce01ffd9ae None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:00.095309 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefef3d3-f2a0-4608-81ab-65ce01ffd9ae None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:00.095470 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefef3d3-f2a0-4608-81ab-65ce01ffd9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:00.113035 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-cefef3d3-f2a0-4608-81ab-65ce01ffd9ae None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:00.114235 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-cefef3d3-f2a0-4608-81ab-65ce01ffd9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:00.122568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 22 11:39:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:39:00.273909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed60dec-ebc3-4641-8707-d0fe01b9b582 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:39:00.288807 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed60dec-ebc3-4641-8707-d0fe01b9b582 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:39:00.294079 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505bc2c-ff59-437a-b3bb-6384ea3028d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:39:00.294709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505bc2c-ff59-437a-b3bb-6384ea3028d0 service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:00.294989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505bc2c-ff59-437a-b3bb-6384ea3028d0 service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:00.295264 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505bc2c-ff59-437a-b3bb-6384ea3028d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:00.303890 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505bc2c-ff59-437a-b3bb-6384ea3028d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:39:00.305465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505bc2c-ff59-437a-b3bb-6384ea3028d0 service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:39:00.314670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 722/1440] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:39:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:39:02.305238 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6973f-e10d-4390-86ec-0716bb9cc3c7 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:02.305798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6973f-e10d-4390-86ec-0716bb9cc3c7 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:02.305798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6973f-e10d-4390-86ec-0716bb9cc3c7 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:02.307181 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 22 11:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:39:02.312393 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4b15c-ef1e-410a-b903-a40c517d297d None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:02.312828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4b15c-ef1e-410a-b903-a40c517d297d None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:02.313066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4b15c-ef1e-410a-b903-a40c517d297d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:02.330946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-8bd4b15c-ef1e-410a-b903-a40c517d297d None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:02.332244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-8bd4b15c-ef1e-410a-b903-a40c517d297d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:02.340089 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 22 11:39:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:39:02.378059 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6973f-e10d-4390-86ec-0716bb9cc3c7 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:39:02.389491 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6973f-e10d-4390-86ec-0716bb9cc3c7 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:39:02.392997 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39b6b6-caf4-4faa-a0d3-041c8b21ae67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:39:02.393621 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39b6b6-caf4-4faa-a0d3-041c8b21ae67 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:02.393898 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39b6b6-caf4-4faa-a0d3-041c8b21ae67 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:02.394169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39b6b6-caf4-4faa-a0d3-041c8b21ae67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:02.403543 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39b6b6-caf4-4faa-a0d3-041c8b21ae67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:39:02.404953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39b6b6-caf4-4faa-a0d3-041c8b21ae67 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:39:02.414813 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 720/1443] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:39:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:39:04.671306 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fb76b-e9b7-4d40-b2e2-d23a3ad7f70b None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:04.671852 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fb76b-e9b7-4d40-b2e2-d23a3ad7f70b None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:04.671972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fb76b-e9b7-4d40-b2e2-d23a3ad7f70b None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:04.673316 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 22 11:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:39:04.679085 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4432d70-857b-4fab-b0c3-6705476467b8 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:04.679499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4432d70-857b-4fab-b0c3-6705476467b8 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:04.679684 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4432d70-857b-4fab-b0c3-6705476467b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:04.697530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-e4432d70-857b-4fab-b0c3-6705476467b8 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:04.698733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e4432d70-857b-4fab-b0c3-6705476467b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:04.706530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 22 11:39:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:39:04.744341 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fb76b-e9b7-4d40-b2e2-d23a3ad7f70b None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:39:04.755052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fb76b-e9b7-4d40-b2e2-d23a3ad7f70b None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:39:04.758554 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a67671-d97c-4e21-9a45-2604139d85ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:39:04.759185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a67671-d97c-4e21-9a45-2604139d85ee service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:04.759505 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a67671-d97c-4e21-9a45-2604139d85ee service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:04.759786 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a67671-d97c-4e21-9a45-2604139d85ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:04.768916 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a67671-d97c-4e21-9a45-2604139d85ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:39:04.770340 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a67671-d97c-4e21-9a45-2604139d85ee service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:39:04.779704 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 725/1446] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:39:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:39:07.110377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f14f2-cd86-4a2a-9dff-493ea14d27f5 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:07.110931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f14f2-cd86-4a2a-9dff-493ea14d27f5 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:07.110931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f14f2-cd86-4a2a-9dff-493ea14d27f5 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:07.112397 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 22 11:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:39:07.118129 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca7760-324e-4e05-a1cd-e793a0a876e3 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:07.118570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca7760-324e-4e05-a1cd-e793a0a876e3 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:07.118718 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca7760-324e-4e05-a1cd-e793a0a876e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:07.136202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d0ca7760-324e-4e05-a1cd-e793a0a876e3 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:07.137404 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d0ca7760-324e-4e05-a1cd-e793a0a876e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:07.145483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 22 11:39:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:39:07.184464 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f14f2-cd86-4a2a-9dff-493ea14d27f5 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:39:07.195461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f14f2-cd86-4a2a-9dff-493ea14d27f5 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:39:07.198920 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc01018-a82f-424b-a810-e624526f111f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:39:07.199595 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc01018-a82f-424b-a810-e624526f111f service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:07.199878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc01018-a82f-424b-a810-e624526f111f service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:07.200152 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc01018-a82f-424b-a810-e624526f111f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:07.209130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc01018-a82f-424b-a810-e624526f111f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:39:07.210557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc01018-a82f-424b-a810-e624526f111f service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:39:07.219849 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 723/1449] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:39:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:39:09.398921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05d609-b3a0-498e-b752-3155e5944429 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:09.399461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05d609-b3a0-498e-b752-3155e5944429 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:09.399595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05d609-b3a0-498e-b752-3155e5944429 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:09.401139 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 22 11:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:39:09.406426 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bbdc4-b250-4309-a2ce-3255fd4876ae None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:09.406839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bbdc4-b250-4309-a2ce-3255fd4876ae None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:09.406986 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bbdc4-b250-4309-a2ce-3255fd4876ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:09.424745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-bd9bbdc4-b250-4309-a2ce-3255fd4876ae None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:09.425939 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-bd9bbdc4-b250-4309-a2ce-3255fd4876ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:09.434443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 22 11:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:39:09.473480 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05d609-b3a0-498e-b752-3155e5944429 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:39:09.484171 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05d609-b3a0-498e-b752-3155e5944429 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:39:09.487659 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9a424-4e87-4ead-8974-e42fc091267a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:39:09.488288 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9a424-4e87-4ead-8974-e42fc091267a service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:09.488572 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9a424-4e87-4ead-8974-e42fc091267a service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:09.488850 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9a424-4e87-4ead-8974-e42fc091267a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:09.497609 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9a424-4e87-4ead-8974-e42fc091267a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:39:09.499030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9a424-4e87-4ead-8974-e42fc091267a service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:39:09.508881 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 728/1452] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:39:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:39:11.650678 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3796bd8-6c34-462b-8e83-432622e35882 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:11.651308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3796bd8-6c34-462b-8e83-432622e35882 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:11.651308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3796bd8-6c34-462b-8e83-432622e35882 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:11.652648 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 22 11:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:39:11.658492 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c2337-12f4-4866-81c9-7791df2cfea4 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:11.659035 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c2337-12f4-4866-81c9-7791df2cfea4 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:11.659129 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c2337-12f4-4866-81c9-7791df2cfea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:11.683912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-1f0c2337-12f4-4866-81c9-7791df2cfea4 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:11.685139 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-1f0c2337-12f4-4866-81c9-7791df2cfea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:11.693732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 22 11:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:39:11.732262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3796bd8-6c34-462b-8e83-432622e35882 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:39:11.743559 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3796bd8-6c34-462b-8e83-432622e35882 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:39:11.747026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43cd82-275b-4d5d-8142-ad55bb3f3168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:39:11.747697 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43cd82-275b-4d5d-8142-ad55bb3f3168 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:11.747975 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43cd82-275b-4d5d-8142-ad55bb3f3168 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:11.748252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43cd82-275b-4d5d-8142-ad55bb3f3168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:11.757611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43cd82-275b-4d5d-8142-ad55bb3f3168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:39:11.759053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43cd82-275b-4d5d-8142-ad55bb3f3168 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:39:11.769152 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 726/1455] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:39:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:39:13.894601 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daf034-28e9-44a5-bcd5-8cd9290b3828 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:13.895218 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daf034-28e9-44a5-bcd5-8cd9290b3828 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:13.895218 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daf034-28e9-44a5-bcd5-8cd9290b3828 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:13.897098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 22 11:39:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:39:13.902678 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9153854-9db7-4107-a171-10dd556b7102 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:13.903183 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9153854-9db7-4107-a171-10dd556b7102 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:13.903300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9153854-9db7-4107-a171-10dd556b7102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:13.923069 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-f9153854-9db7-4107-a171-10dd556b7102 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:13.924388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f9153854-9db7-4107-a171-10dd556b7102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:13.932390 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 22 11:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:39:13.972027 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daf034-28e9-44a5-bcd5-8cd9290b3828 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:39:13.985694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daf034-28e9-44a5-bcd5-8cd9290b3828 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:39:13.989382 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1702754-dc2f-4ca2-a815-28286f81105d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:39:13.990060 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1702754-dc2f-4ca2-a815-28286f81105d service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:13.990348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1702754-dc2f-4ca2-a815-28286f81105d service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:13.990628 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1702754-dc2f-4ca2-a815-28286f81105d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:14.000741 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1702754-dc2f-4ca2-a815-28286f81105d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:39:14.002206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1702754-dc2f-4ca2-a815-28286f81105d service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:39:14.013732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 731/1458] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:39:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:39:16.111163 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df062493-abbd-4c97-8409-2631c48d443f None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:16.111762 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df062493-abbd-4c97-8409-2631c48d443f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:16.111762 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df062493-abbd-4c97-8409-2631c48d443f None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:16.113221 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 22 11:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:39:16.119656 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450342ac-f495-485b-84f3-4ffe7efa4f1e None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:16.120192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450342ac-f495-485b-84f3-4ffe7efa4f1e None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:16.120192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450342ac-f495-485b-84f3-4ffe7efa4f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:16.138639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-450342ac-f495-485b-84f3-4ffe7efa4f1e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:16.140278 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-450342ac-f495-485b-84f3-4ffe7efa4f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:16.150116 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 22 11:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:39:16.188805 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df062493-abbd-4c97-8409-2631c48d443f None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:39:16.199873 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df062493-abbd-4c97-8409-2631c48d443f None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:39:16.203403 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d21f6-a57c-42ec-9e22-e46226613ad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:39:16.204053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d21f6-a57c-42ec-9e22-e46226613ad4 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:16.204351 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d21f6-a57c-42ec-9e22-e46226613ad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:16.204628 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d21f6-a57c-42ec-9e22-e46226613ad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:16.214902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d21f6-a57c-42ec-9e22-e46226613ad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:39:16.216382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d21f6-a57c-42ec-9e22-e46226613ad4 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:39:16.226899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 729/1461] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:39:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:39:18.408441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd18e2-2e27-48bb-95e1-8c58a5818e46 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:18.408987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd18e2-2e27-48bb-95e1-8c58a5818e46 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:18.408987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd18e2-2e27-48bb-95e1-8c58a5818e46 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:18.410355 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 22 11:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:39:18.416338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d84e84-d131-4290-9391-821485cd66d3 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:18.416869 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d84e84-d131-4290-9391-821485cd66d3 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:18.416964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d84e84-d131-4290-9391-821485cd66d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:18.423135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=79 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:39:18.443563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-f8d84e84-d131-4290-9391-821485cd66d3 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:18.444781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f8d84e84-d131-4290-9391-821485cd66d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:18.452867 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 22 11:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:39:18.495512 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd18e2-2e27-48bb-95e1-8c58a5818e46 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:39:18.501811 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=73 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:39:18.509235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd18e2-2e27-48bb-95e1-8c58a5818e46 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:39:18.512979 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76ae65-bd02-41ea-86e9-93a0d0adc485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:39:18.513889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76ae65-bd02-41ea-86e9-93a0d0adc485 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:18.514162 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76ae65-bd02-41ea-86e9-93a0d0adc485 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:18.514435 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76ae65-bd02-41ea-86e9-93a0d0adc485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:18.524778 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76ae65-bd02-41ea-86e9-93a0d0adc485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:39:18.526233 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76ae65-bd02-41ea-86e9-93a0d0adc485 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:39:18.536903 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 734/1464] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:39:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:39:20.668239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe869f-07eb-4863-aa9e-7923a4d1ca33 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:20.668843 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe869f-07eb-4863-aa9e-7923a4d1ca33 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:20.668843 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe869f-07eb-4863-aa9e-7923a4d1ca33 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:20.670182 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 22 11:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:39:20.675816 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c31d56-5e21-407c-beb9-74bc77c62038 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:20.676254 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c31d56-5e21-407c-beb9-74bc77c62038 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:20.676376 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c31d56-5e21-407c-beb9-74bc77c62038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:20.695910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d4c31d56-5e21-407c-beb9-74bc77c62038 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:20.697147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d4c31d56-5e21-407c-beb9-74bc77c62038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:20.705288 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 22 11:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:39:20.743063 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe869f-07eb-4863-aa9e-7923a4d1ca33 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:39:20.744834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-74fe869f-07eb-4863-aa9e-7923a4d1ca33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:20.758300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe869f-07eb-4863-aa9e-7923a4d1ca33 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:39:20.761813 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724f675-3e8f-4578-b598-f7aa2a50dd99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:39:20.762462 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724f675-3e8f-4578-b598-f7aa2a50dd99 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:20.762738 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724f675-3e8f-4578-b598-f7aa2a50dd99 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:20.763017 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724f675-3e8f-4578-b598-f7aa2a50dd99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:20.772888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b724f675-3e8f-4578-b598-f7aa2a50dd99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:39:20.774318 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b724f675-3e8f-4578-b598-f7aa2a50dd99 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:39:20.783987 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 732/1467] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:39:22.882589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375fa9e-e79b-4eda-beea-046dadd5530f None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:22.883118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375fa9e-e79b-4eda-beea-046dadd5530f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:22.883245 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375fa9e-e79b-4eda-beea-046dadd5530f None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:22.884555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 22 11:39:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:39:22.890092 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d174a-999d-4cbf-a255-38c95716e51f None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:22.890514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d174a-999d-4cbf-a255-38c95716e51f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:22.890652 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d174a-999d-4cbf-a255-38c95716e51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:22.909109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-9b6d174a-999d-4cbf-a255-38c95716e51f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:22.910355 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-9b6d174a-999d-4cbf-a255-38c95716e51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:22.918380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 22 11:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:39:22.955893 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375fa9e-e79b-4eda-beea-046dadd5530f None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:39:22.966955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375fa9e-e79b-4eda-beea-046dadd5530f None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:39:22.970468 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d520c-b66b-4d75-bf53-5668022427d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:39:22.971110 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d520c-b66b-4d75-bf53-5668022427d6 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:22.971424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d520c-b66b-4d75-bf53-5668022427d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:22.971705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d520c-b66b-4d75-bf53-5668022427d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:22.982147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d520c-b66b-4d75-bf53-5668022427d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:39:22.983619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d520c-b66b-4d75-bf53-5668022427d6 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:39:22.993836 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 737/1470] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:39:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:39:32.903626 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=11 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:39:32.989000 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=14 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:39:43.185333 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235724f-b62f-4b06-8de8-b24b2be18347 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:43.185920 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235724f-b62f-4b06-8de8-b24b2be18347 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:43.185920 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235724f-b62f-4b06-8de8-b24b2be18347 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:43.187263 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 734/1471] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:39:43.192638 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1d567-acbf-4e36-a464-82a42a65e78e None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:43.193639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1d567-acbf-4e36-a464-82a42a65e78e None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:43.193792 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1d567-acbf-4e36-a464-82a42a65e78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:43.211012 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-8cb1d567-acbf-4e36-a464-82a42a65e78e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:43.212235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-8cb1d567-acbf-4e36-a464-82a42a65e78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:43.220004 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 738/1472] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:39:43.268425 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d8c29-41f9-46d7-b39b-12fdc0d165c8 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:43.268716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d8c29-41f9-46d7-b39b-12fdc0d165c8 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:43.269000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d8c29-41f9-46d7-b39b-12fdc0d165c8 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:43.270366 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 735/1473] 10.222.0.36 () {54 vars in 962 bytes} [Wed Jun 22 11:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:39:43.275187 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d507c6-e5f4-44c4-b907-47cd957445ac None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:43.275503 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d507c6-e5f4-44c4-b907-47cd957445ac None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:43.275785 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d507c6-e5f4-44c4-b907-47cd957445ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:43.293454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-61d507c6-e5f4-44c4-b907-47cd957445ac None None] MFA Rules not processed for user `7ed133721ed044708ee362c93f564d07`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:43.294636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-61d507c6-e5f4-44c4-b907-47cd957445ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:43.301121 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 739/1474] 10.222.0.36 () {58 vars in 1059 bytes} [Wed Jun 22 11:39:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:39:43.308839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d8c29-41f9-46d7-b39b-12fdc0d165c8 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:39:43.319917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d8c29-41f9-46d7-b39b-12fdc0d165c8 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:39:43.323500 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd38b78-6cb7-455f-b8de-7972765d64a2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ed133721ed044708ee362c93f564d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:39:43.324131 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd38b78-6cb7-455f-b8de-7972765d64a2 service glance] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:43.324413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd38b78-6cb7-455f-b8de-7972765d64a2 service glance] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:43.324688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd38b78-6cb7-455f-b8de-7972765d64a2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:43.333474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd38b78-6cb7-455f-b8de-7972765d64a2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:39:43.334887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd38b78-6cb7-455f-b8de-7972765d64a2 service glance] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:39:43.344105 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 736/1475] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:39:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:39:45.494339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dc499-c9a2-4f9d-a860-6614d94010f6 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:45.494904 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dc499-c9a2-4f9d-a860-6614d94010f6 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:45.494904 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dc499-c9a2-4f9d-a860-6614d94010f6 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:45.496287 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 740/1476] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:39:45.501398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df2f98-f0b3-47d2-a412-46726c61db1e None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:45.501835 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df2f98-f0b3-47d2-a412-46726c61db1e None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:45.501965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df2f98-f0b3-47d2-a412-46726c61db1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:45.519762 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c8df2f98-f0b3-47d2-a412-46726c61db1e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:45.521499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c8df2f98-f0b3-47d2-a412-46726c61db1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:45.529567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 737/1477] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:39:45.553545 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dc499-c9a2-4f9d-a860-6614d94010f6 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:39:45.565936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dc499-c9a2-4f9d-a860-6614d94010f6 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:39:45.570475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75702aa-88ff-4280-91d9-867fc7cd3d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:39:45.571118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75702aa-88ff-4280-91d9-867fc7cd3d8b admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:45.571426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75702aa-88ff-4280-91d9-867fc7cd3d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:45.571706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75702aa-88ff-4280-91d9-867fc7cd3d8b admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:45.572433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75702aa-88ff-4280-91d9-867fc7cd3d8b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:39:45.574238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: WARNING py.warnings [None req-b75702aa-88ff-4280-91d9-867fc7cd3d8b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 22 11:39:45.574238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: warnings.warn(msg) Jun 22 11:39:45.574238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: Jun 22 11:39:45.575392 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75702aa-88ff-4280-91d9-867fc7cd3d8b admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:39:45.595414 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 741/1478] 10.222.0.36 () {56 vars in 1108 bytes} [Wed Jun 22 11:39:45 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:39:47.660603 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566e5ff-613e-4565-9742-8e25b6d4f5f5 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:47.661165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566e5ff-613e-4565-9742-8e25b6d4f5f5 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:47.661165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566e5ff-613e-4565-9742-8e25b6d4f5f5 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:47.663403 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 738/1479] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:39:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:39:47.668917 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c6998-f791-4dc0-828f-3fcd1e94c846 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:47.669436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c6998-f791-4dc0-828f-3fcd1e94c846 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:47.669555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c6998-f791-4dc0-828f-3fcd1e94c846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:47.687843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-190c6998-f791-4dc0-828f-3fcd1e94c846 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:47.689091 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-190c6998-f791-4dc0-828f-3fcd1e94c846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:47.697224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 742/1480] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:39:47.744030 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566e5ff-613e-4565-9742-8e25b6d4f5f5 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:39:47.763321 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566e5ff-613e-4565-9742-8e25b6d4f5f5 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:39:47.770059 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe11769-862c-4a21-94ec-5c6fa9468fc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:39:47.771392 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe11769-862c-4a21-94ec-5c6fa9468fc5 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:47.771809 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe11769-862c-4a21-94ec-5c6fa9468fc5 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:47.772196 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe11769-862c-4a21-94ec-5c6fa9468fc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:47.782586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe11769-862c-4a21-94ec-5c6fa9468fc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:39:47.784243 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe11769-862c-4a21-94ec-5c6fa9468fc5 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:39:47.795613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 739/1481] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:39:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:39:50.038069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ddeb4-2927-4d29-952c-df299a161178 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:50.038602 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ddeb4-2927-4d29-952c-df299a161178 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:50.038738 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ddeb4-2927-4d29-952c-df299a161178 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:50.040092 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 743/1482] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:39:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:39:50.045293 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0867c-0f48-4619-a2e3-235b8992d91c None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:50.045731 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0867c-0f48-4619-a2e3-235b8992d91c None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:50.045832 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0867c-0f48-4619-a2e3-235b8992d91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:50.063188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-82e0867c-0f48-4619-a2e3-235b8992d91c None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:50.064412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-82e0867c-0f48-4619-a2e3-235b8992d91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:50.072076 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 740/1483] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:39:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:39:50.108292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ddeb4-2927-4d29-952c-df299a161178 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:39:50.119512 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ddeb4-2927-4d29-952c-df299a161178 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:39:50.122978 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800bbea-7d58-4b1e-9e04-f799d13829fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:39:50.123648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800bbea-7d58-4b1e-9e04-f799d13829fa service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:50.123930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800bbea-7d58-4b1e-9e04-f799d13829fa service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:50.124204 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800bbea-7d58-4b1e-9e04-f799d13829fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:50.133988 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800bbea-7d58-4b1e-9e04-f799d13829fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:39:50.135452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800bbea-7d58-4b1e-9e04-f799d13829fa service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:39:50.144988 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 744/1484] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:39:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:39:52.244201 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092c261-df0c-4556-b709-ccc55ea44480 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:52.244751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092c261-df0c-4556-b709-ccc55ea44480 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:52.244751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092c261-df0c-4556-b709-ccc55ea44480 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:52.246109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 741/1485] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:39:52.252065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4d7c9-f79a-48e9-a312-f135b83c553d None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:52.252528 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4d7c9-f79a-48e9-a312-f135b83c553d None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:52.252642 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d4d7c9-f79a-48e9-a312-f135b83c553d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:52.271108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-51d4d7c9-f79a-48e9-a312-f135b83c553d None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:52.272341 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-51d4d7c9-f79a-48e9-a312-f135b83c553d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:52.280217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 745/1486] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:39:52.318514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092c261-df0c-4556-b709-ccc55ea44480 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:39:52.329287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e092c261-df0c-4556-b709-ccc55ea44480 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:39:52.332783 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52bfcd-6176-417b-b8ea-d66a609a2aa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:39:52.333430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52bfcd-6176-417b-b8ea-d66a609a2aa2 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:52.333711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52bfcd-6176-417b-b8ea-d66a609a2aa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:52.333989 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52bfcd-6176-417b-b8ea-d66a609a2aa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:52.343578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52bfcd-6176-417b-b8ea-d66a609a2aa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:39:52.345016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca52bfcd-6176-417b-b8ea-d66a609a2aa2 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:39:52.354880 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 742/1487] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:39:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:39:54.595313 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cd11e-2cb4-4212-ba30-37e5e0d5e8e7 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:54.595916 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cd11e-2cb4-4212-ba30-37e5e0d5e8e7 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:54.595916 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cd11e-2cb4-4212-ba30-37e5e0d5e8e7 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:54.597271 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 746/1488] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:39:54.602833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a96fff-52d4-46ec-8523-6c36d082330d None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:54.603296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a96fff-52d4-46ec-8523-6c36d082330d None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:54.603449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a96fff-52d4-46ec-8523-6c36d082330d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:54.622402 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c9a96fff-52d4-46ec-8523-6c36d082330d None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:54.623647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c9a96fff-52d4-46ec-8523-6c36d082330d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:54.632334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 743/1489] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:39:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:39:54.673492 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cd11e-2cb4-4212-ba30-37e5e0d5e8e7 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:39:54.686250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cd11e-2cb4-4212-ba30-37e5e0d5e8e7 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:39:54.689863 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f55a7-17c1-4427-ad08-6a3a19c3b024 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:39:54.690507 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f55a7-17c1-4427-ad08-6a3a19c3b024 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:54.690792 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f55a7-17c1-4427-ad08-6a3a19c3b024 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:54.691069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f55a7-17c1-4427-ad08-6a3a19c3b024 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:54.705941 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f55a7-17c1-4427-ad08-6a3a19c3b024 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:39:54.707504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972f55a7-17c1-4427-ad08-6a3a19c3b024 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:39:54.717793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 747/1490] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:39:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:39:56.967548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db15d0e-6d9d-48bb-984b-fdd6ef3e28c6 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:56.968056 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db15d0e-6d9d-48bb-984b-fdd6ef3e28c6 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:56.968056 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db15d0e-6d9d-48bb-984b-fdd6ef3e28c6 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:56.969458 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 744/1491] 10.222.0.36 () {54 vars in 926 bytes} [Wed Jun 22 11:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:39:56.974564 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0dc2c-24c7-46e4-a7d7-16a8dfd5b665 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:56.975005 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0dc2c-24c7-46e4-a7d7-16a8dfd5b665 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:56.975132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0dc2c-24c7-46e4-a7d7-16a8dfd5b665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:56.993113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-38d0dc2c-24c7-46e4-a7d7-16a8dfd5b665 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:39:56.994306 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-38d0dc2c-24c7-46e4-a7d7-16a8dfd5b665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:39:57.002102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 748/1492] 10.222.0.36 () {58 vars in 1023 bytes} [Wed Jun 22 11:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:39:57.041998 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db15d0e-6d9d-48bb-984b-fdd6ef3e28c6 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:39:57.053445 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db15d0e-6d9d-48bb-984b-fdd6ef3e28c6 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:39:57.057007 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230aa709-769c-4712-b10a-dd27f00cf962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:39:57.057638 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230aa709-769c-4712-b10a-dd27f00cf962 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:39:57.057921 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230aa709-769c-4712-b10a-dd27f00cf962 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:39:57.058196 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230aa709-769c-4712-b10a-dd27f00cf962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:39:57.067131 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230aa709-769c-4712-b10a-dd27f00cf962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:39:57.068654 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230aa709-769c-4712-b10a-dd27f00cf962 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:39:57.078412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 745/1493] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:39:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:40:06.987823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=33 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:40:07.074050 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=33 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:41:24.537311 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3f61b-f6eb-4503-9c80-df968c48a7af None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:41:24.537910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3f61b-f6eb-4503-9c80-df968c48a7af None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:41:24.537910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3f61b-f6eb-4503-9c80-df968c48a7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:41:24.555965 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f1c3f61b-f6eb-4503-9c80-df968c48a7af None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:41:24.557161 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f1c3f61b-f6eb-4503-9c80-df968c48a7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:41:24.564981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 749/1494] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:41:24.574736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff0b96-9839-4912-a572-ae33356f77f9 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:41:24.575227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff0b96-9839-4912-a572-ae33356f77f9 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:41:24.575322 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff0b96-9839-4912-a572-ae33356f77f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:41:24.593461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-5aff0b96-9839-4912-a572-ae33356f77f9 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:41:24.594653 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-5aff0b96-9839-4912-a572-ae33356f77f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:41:24.602629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 746/1495] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:41:24.607160 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3f61b-f6eb-4503-9c80-df968c48a7af None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:41:24.617471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3f61b-f6eb-4503-9c80-df968c48a7af None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:41:24.621963 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8274e-929c-4b93-aab6-30f359ba2ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:41:24.622618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8274e-929c-4b93-aab6-30f359ba2ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:41:24.622903 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8274e-929c-4b93-aab6-30f359ba2ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:41:24.623190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8274e-929c-4b93-aab6-30f359ba2ee3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:41:24.623954 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8274e-929c-4b93-aab6-30f359ba2ee3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:41:24.624261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8274e-929c-4b93-aab6-30f359ba2ee3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:41:24.625284 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: WARNING py.warnings [None req-d3f8274e-929c-4b93-aab6-30f359ba2ee3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 22 11:41:24.625284 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: warnings.warn(msg) Jun 22 11:41:24.625284 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: Jun 22 11:41:24.626166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8274e-929c-4b93-aab6-30f359ba2ee3 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:41:24.638603 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 750/1496] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:41:24.647012 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff0b96-9839-4912-a572-ae33356f77f9 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:41:24.656123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff0b96-9839-4912-a572-ae33356f77f9 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:41:24.661219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63665f26-c56e-4906-86bb-2c5167714744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:41:24.661861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63665f26-c56e-4906-86bb-2c5167714744 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:41:24.662140 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63665f26-c56e-4906-86bb-2c5167714744 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:41:24.662412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63665f26-c56e-4906-86bb-2c5167714744 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:41:24.663088 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63665f26-c56e-4906-86bb-2c5167714744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:41:24.664908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: WARNING py.warnings [None req-63665f26-c56e-4906-86bb-2c5167714744 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 22 11:41:24.664908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: warnings.warn(msg) Jun 22 11:41:24.664908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: Jun 22 11:41:24.666084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63665f26-c56e-4906-86bb-2c5167714744 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:41:24.702349 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 747/1497] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:41:24 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:41:24.706490 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8274e-929c-4b93-aab6-30f359ba2ee3 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:41:24.716312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8274e-929c-4b93-aab6-30f359ba2ee3 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:41:24.773008 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d77040-bd6f-4ddf-90a9-a01127fbbd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:41:24.773702 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d77040-bd6f-4ddf-90a9-a01127fbbd53 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:41:24.774001 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d77040-bd6f-4ddf-90a9-a01127fbbd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:41:24.774263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d77040-bd6f-4ddf-90a9-a01127fbbd53 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:41:24.774994 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d77040-bd6f-4ddf-90a9-a01127fbbd53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:41:24.776057 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: WARNING py.warnings [None req-c5d77040-bd6f-4ddf-90a9-a01127fbbd53 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 22 11:41:24.776057 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: warnings.warn(msg) Jun 22 11:41:24.776057 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: Jun 22 11:41:24.777291 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d77040-bd6f-4ddf-90a9-a01127fbbd53 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:41:24.798006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 751/1498] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:41:24 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:41:24.803213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63665f26-c56e-4906-86bb-2c5167714744 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:41:24.814009 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63665f26-c56e-4906-86bb-2c5167714744 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:41:24.818609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b92df-3795-499a-9100-de9b068bd740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:41:24.819269 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b92df-3795-499a-9100-de9b068bd740 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:41:24.819581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b92df-3795-499a-9100-de9b068bd740 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:41:24.819859 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b92df-3795-499a-9100-de9b068bd740 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:41:24.820589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b92df-3795-499a-9100-de9b068bd740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:41:24.821599 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: WARNING py.warnings [None req-d75b92df-3795-499a-9100-de9b068bd740 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 22 11:41:24.821599 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: warnings.warn(msg) Jun 22 11:41:24.821599 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: Jun 22 11:41:24.822486 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b92df-3795-499a-9100-de9b068bd740 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:41:24.835730 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 748/1499] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:41:24.839950 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d77040-bd6f-4ddf-90a9-a01127fbbd53 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:41:24.850592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d77040-bd6f-4ddf-90a9-a01127fbbd53 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:41:24.855065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ede78-e8b6-46e2-a792-96df908f3fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:41:24.855753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ede78-e8b6-46e2-a792-96df908f3fbf admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:41:24.856040 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ede78-e8b6-46e2-a792-96df908f3fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:41:24.856319 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ede78-e8b6-46e2-a792-96df908f3fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:41:24.857043 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ede78-e8b6-46e2-a792-96df908f3fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:41:24.858060 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: WARNING py.warnings [None req-fb4ede78-e8b6-46e2-a792-96df908f3fbf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 22 11:41:24.858060 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: warnings.warn(msg) Jun 22 11:41:24.858060 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: Jun 22 11:41:24.858947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ede78-e8b6-46e2-a792-96df908f3fbf admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:41:24.872696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 752/1500] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:41:24.876996 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b92df-3795-499a-9100-de9b068bd740 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:41:24.887786 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b92df-3795-499a-9100-de9b068bd740 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:41:24.892658 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01d48b-809e-4443-95ee-3a200d79f878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:41:24.893535 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01d48b-809e-4443-95ee-3a200d79f878 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:41:24.893812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01d48b-809e-4443-95ee-3a200d79f878 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:41:24.894083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01d48b-809e-4443-95ee-3a200d79f878 admin admin] PATH_INFO: `/v3/projects/1ce73af39566442f908779ee8c7fa060/users/bdc2f2846b014270a497df267a7c25e2/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:41:24.908294 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e01d48b-809e-4443-95ee-3a200d79f878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ce73af39566442f908779ee8c7fa060, user_id=bdc2f2846b014270a497df267a7c25e2, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:41:24.909318 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: WARNING py.warnings [None req-7e01d48b-809e-4443-95ee-3a200d79f878 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 22 11:41:24.909318 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: warnings.warn(msg) Jun 22 11:41:24.909318 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: Jun 22 11:41:24.911815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e01d48b-809e-4443-95ee-3a200d79f878 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:41:24.924995 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 749/1501] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:41:24 2022] PUT /identity/v3/projects/1ce73af39566442f908779ee8c7fa060/users/bdc2f2846b014270a497df267a7c25e2/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:41:24.936756 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627f4b1-6790-4193-97d4-0c7a3c9596fe None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:41:24.937047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627f4b1-6790-4193-97d4-0c7a3c9596fe None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:41:24.937327 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627f4b1-6790-4193-97d4-0c7a3c9596fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:41:24.964368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-2627f4b1-6790-4193-97d4-0c7a3c9596fe None None] MFA Rules not processed for user `bdc2f2846b014270a497df267a7c25e2`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:41:24.965590 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-2627f4b1-6790-4193-97d4-0c7a3c9596fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:41:25.028295 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 753/1502] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:41:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:41:25.042170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048635be-3153-4fdf-b0ba-af3712789da9 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:41:25.042458 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048635be-3153-4fdf-b0ba-af3712789da9 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:41:25.042737 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048635be-3153-4fdf-b0ba-af3712789da9 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:41:25.044137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 750/1503] 10.222.0.36 () {54 vars in 963 bytes} [Wed Jun 22 11:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:41:25.049025 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae02f90-8857-4a89-9c89-939b446bf48f None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:41:25.049257 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae02f90-8857-4a89-9c89-939b446bf48f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:41:25.049536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae02f90-8857-4a89-9c89-939b446bf48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:41:25.068030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-fae02f90-8857-4a89-9c89-939b446bf48f None None] MFA Rules not processed for user `ceb170ac6353495b9d1dc6723d59567a`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:41:25.069229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-fae02f90-8857-4a89-9c89-939b446bf48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:41:25.111644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:41:25.120594 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 754/1504] 10.222.0.36 () {58 vars in 1060 bytes} [Wed Jun 22 11:41:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:41:25.128325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048635be-3153-4fdf-b0ba-af3712789da9 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:41:25.139472 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048635be-3153-4fdf-b0ba-af3712789da9 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:41:25.143200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71befcff-65e1-40d7-9adb-7872c4bc14a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:41:25.143931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71befcff-65e1-40d7-9adb-7872c4bc14a2 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:41:25.144212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71befcff-65e1-40d7-9adb-7872c4bc14a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:41:25.144489 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71befcff-65e1-40d7-9adb-7872c4bc14a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:41:25.152535 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71befcff-65e1-40d7-9adb-7872c4bc14a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:41:25.153948 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71befcff-65e1-40d7-9adb-7872c4bc14a2 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:41:25.162603 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 751/1505] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:41:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:41:25.204395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b746df5-69b3-4b83-81f5-68b1f3701110 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:41:25.204689 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b746df5-69b3-4b83-81f5-68b1f3701110 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:41:25.204997 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b746df5-69b3-4b83-81f5-68b1f3701110 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:41:25.206402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 755/1506] 10.222.0.36 () {52 vars in 829 bytes} [Wed Jun 22 11:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:41:25.210033 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71befcff-65e1-40d7-9adb-7872c4bc14a2 service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:41:25.270867 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71befcff-65e1-40d7-9adb-7872c4bc14a2 service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:41:25.275609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56274ac5-ff12-4c99-b480-903d73f9ae8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:41:25.276473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56274ac5-ff12-4c99-b480-903d73f9ae8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:41:25.276752 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56274ac5-ff12-4c99-b480-903d73f9ae8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:41:25.277023 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56274ac5-ff12-4c99-b480-903d73f9ae8b admin admin] PATH_INFO: `/v3/users/bdc2f2846b014270a497df267a7c25e2` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:41:25.278194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56274ac5-ff12-4c99-b480-903d73f9ae8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdc2f2846b014270a497df267a7c25e2)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:41:25.279214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: WARNING py.warnings [None req-56274ac5-ff12-4c99-b480-903d73f9ae8b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 22 11:41:25.279214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: warnings.warn(msg) Jun 22 11:41:25.279214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: Jun 22 11:41:25.280441 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56274ac5-ff12-4c99-b480-903d73f9ae8b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:41:25.345243 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-56274ac5-ff12-4c99-b480-903d73f9ae8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdc2f2846b014270a497df267a7c25e2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:41:25.363285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-56274ac5-ff12-4c99-b480-903d73f9ae8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdc2f2846b014270a497df267a7c25e2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:41:25.373278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-56274ac5-ff12-4c99-b480-903d73f9ae8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdc2f2846b014270a497df267a7c25e2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:41:25.374223 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-56274ac5-ff12-4c99-b480-903d73f9ae8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdc2f2846b014270a497df267a7c25e2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:41:25.383767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:41:25.387924 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-56274ac5-ff12-4c99-b480-903d73f9ae8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdc2f2846b014270a497df267a7c25e2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:41:25.396189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 752/1507] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:41:25 2022] DELETE /identity/v3/users/bdc2f2846b014270a497df267a7c25e2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:41:25.400378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0be276-5f72-47c1-ad5c-e5e4a68f756b None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:41:25.400667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0be276-5f72-47c1-ad5c-e5e4a68f756b None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:41:25.400944 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0be276-5f72-47c1-ad5c-e5e4a68f756b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:41:25.419090 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-ca0be276-5f72-47c1-ad5c-e5e4a68f756b None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:41:25.420333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ca0be276-5f72-47c1-ad5c-e5e4a68f756b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:41:25.480327 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 756/1508] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:41:25.491826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56274ac5-ff12-4c99-b480-903d73f9ae8b admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:41:25.493248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-56274ac5-ff12-4c99-b480-903d73f9ae8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:41:25.550953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56274ac5-ff12-4c99-b480-903d73f9ae8b admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:41:25.555134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b14fd3-d1f0-4f82-8127-9efd73665e3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:41:25.555816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b14fd3-d1f0-4f82-8127-9efd73665e3a service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:41:25.556100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b14fd3-d1f0-4f82-8127-9efd73665e3a service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:41:25.556374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b14fd3-d1f0-4f82-8127-9efd73665e3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:41:25.565191 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b14fd3-d1f0-4f82-8127-9efd73665e3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:41:25.566611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b14fd3-d1f0-4f82-8127-9efd73665e3a service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:41:25.575876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 753/1509] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:41:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:41:25.907302 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0be276-5f72-47c1-ad5c-e5e4a68f756b None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:41:25.909101 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ca0be276-5f72-47c1-ad5c-e5e4a68f756b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:41:25.921535 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0be276-5f72-47c1-ad5c-e5e4a68f756b None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:41:25.930342 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002c69f-a5f3-408b-b0ad-318a64919370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:41:25.930342 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002c69f-a5f3-408b-b0ad-318a64919370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:41:25.930342 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002c69f-a5f3-408b-b0ad-318a64919370 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:41:25.930342 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002c69f-a5f3-408b-b0ad-318a64919370 admin admin] PATH_INFO: `/v3/projects/1ce73af39566442f908779ee8c7fa060` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:41:25.930894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002c69f-a5f3-408b-b0ad-318a64919370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ce73af39566442f908779ee8c7fa060)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:41:25.931025 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: WARNING py.warnings [None req-f002c69f-a5f3-408b-b0ad-318a64919370 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 22 11:41:25.931025 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: warnings.warn(msg) Jun 22 11:41:25.931025 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: Jun 22 11:41:25.932279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002c69f-a5f3-408b-b0ad-318a64919370 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:41:25.996807 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f002c69f-a5f3-408b-b0ad-318a64919370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ce73af39566442f908779ee8c7fa060', 'request_id': 'req-f002c69f-a5f3-408b-b0ad-318a64919370'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:41:26.011005 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f002c69f-a5f3-408b-b0ad-318a64919370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ce73af39566442f908779ee8c7fa060', 'request_id': 'req-f002c69f-a5f3-408b-b0ad-318a64919370'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:41:26.021373 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f002c69f-a5f3-408b-b0ad-318a64919370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ce73af39566442f908779ee8c7fa060', 'request_id': 'req-f002c69f-a5f3-408b-b0ad-318a64919370'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:41:26.032686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f002c69f-a5f3-408b-b0ad-318a64919370 admin admin] The token cache is being invalidate because project 1ce73af39566442f908779ee8c7fa060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:41:26.032981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f002c69f-a5f3-408b-b0ad-318a64919370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:41:26.033710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f002c69f-a5f3-408b-b0ad-318a64919370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:41:26.035266 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 757/1510] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:41:25 2022] DELETE /identity/v3/projects/1ce73af39566442f908779ee8c7fa060 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:41:35.571865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=29 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:41:36.029738 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:43:06.078953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d80102-08b8-4ff5-95fd-9d6837d4db8c None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:06.079610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d80102-08b8-4ff5-95fd-9d6837d4db8c None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:06.079610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d80102-08b8-4ff5-95fd-9d6837d4db8c None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:06.080935 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 22 11:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:43:06.086330 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b0e51-3596-4fc7-8b66-a59518d5cd62 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:06.086803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b0e51-3596-4fc7-8b66-a59518d5cd62 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:06.086911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b0e51-3596-4fc7-8b66-a59518d5cd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:06.105292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-b47b0e51-3596-4fc7-8b66-a59518d5cd62 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:43:06.106494 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b47b0e51-3596-4fc7-8b66-a59518d5cd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:43:06.165067 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 22 11:43:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:43:06.320672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d80102-08b8-4ff5-95fd-9d6837d4db8c None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:43:06.322772 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-42d80102-08b8-4ff5-95fd-9d6837d4db8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:43:06.398550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d80102-08b8-4ff5-95fd-9d6837d4db8c None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:43:06.403953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd733343-9156-434b-a501-f32999a0eeab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:43:06.404607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd733343-9156-434b-a501-f32999a0eeab service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:06.404905 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd733343-9156-434b-a501-f32999a0eeab service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:06.405184 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd733343-9156-434b-a501-f32999a0eeab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:06.413305 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd733343-9156-434b-a501-f32999a0eeab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:43:06.414888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd733343-9156-434b-a501-f32999a0eeab service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:43:06.424097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 755/1513] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:43:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:43:07.259197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab41044-47d1-4974-bc7a-d82faf28cc65 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:07.259817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab41044-47d1-4974-bc7a-d82faf28cc65 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:07.259817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab41044-47d1-4974-bc7a-d82faf28cc65 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:07.261142 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 22 11:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:43:07.266060 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd9bb3-c04b-4c54-bbeb-aa3e5aebdadc None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:07.266529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd9bb3-c04b-4c54-bbeb-aa3e5aebdadc None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:07.266634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd9bb3-c04b-4c54-bbeb-aa3e5aebdadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:07.285268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-17fd9bb3-c04b-4c54-bbeb-aa3e5aebdadc None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:43:07.286472 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-17fd9bb3-c04b-4c54-bbeb-aa3e5aebdadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:43:07.294335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 22 11:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:43:07.450109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab41044-47d1-4974-bc7a-d82faf28cc65 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:43:07.465079 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab41044-47d1-4974-bc7a-d82faf28cc65 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:43:07.470499 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b736b6-0af5-4f61-9f06-2039a4c535f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:43:07.471127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b736b6-0af5-4f61-9f06-2039a4c535f2 service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:07.471437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b736b6-0af5-4f61-9f06-2039a4c535f2 service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:07.471713 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b736b6-0af5-4f61-9f06-2039a4c535f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:07.479960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b736b6-0af5-4f61-9f06-2039a4c535f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:43:07.481553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b736b6-0af5-4f61-9f06-2039a4c535f2 service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:43:07.490846 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 760/1516] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:43:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:43:08.308621 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1edfa5-fb52-46fb-8847-cfe7e12e14d4 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:08.309130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1edfa5-fb52-46fb-8847-cfe7e12e14d4 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:08.309130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1edfa5-fb52-46fb-8847-cfe7e12e14d4 None None] PATH_INFO: `/v3` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:08.310399 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 757/1517] 10.222.0.62 () {54 vars in 925 bytes} [Wed Jun 22 11:43:08 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:43:08.316494 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07761c78-0160-425f-a0a0-14f79e512238 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:08.316935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07761c78-0160-425f-a0a0-14f79e512238 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:08.317058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07761c78-0160-425f-a0a0-14f79e512238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:08.335618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-07761c78-0160-425f-a0a0-14f79e512238 None None] MFA Rules not processed for user `93630ed5e6dc4a5f958bc7cf1a68a26c`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:43:08.336833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-07761c78-0160-425f-a0a0-14f79e512238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:43:08.398335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 761/1518] 10.222.0.62 () {58 vars in 1015 bytes} [Wed Jun 22 11:43:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:43:17.280192 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=37 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:43:17.353906 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bdc1d-58b5-4ce2-816c-34ca5042e9aa None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:17.354103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bdc1d-58b5-4ce2-816c-34ca5042e9aa None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:17.354384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bdc1d-58b5-4ce2-816c-34ca5042e9aa None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:17.355844 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 758/1519] 10.222.0.36 () {54 vars in 960 bytes} [Wed Jun 22 11:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:43:17.361072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7117b03-2884-4f1a-a2c6-f58c6380ba9d None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:17.361546 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7117b03-2884-4f1a-a2c6-f58c6380ba9d None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:17.361654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7117b03-2884-4f1a-a2c6-f58c6380ba9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:17.380575 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-a7117b03-2884-4f1a-a2c6-f58c6380ba9d None None] MFA Rules not processed for user `93630ed5e6dc4a5f958bc7cf1a68a26c`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:43:17.381793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-a7117b03-2884-4f1a-a2c6-f58c6380ba9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:43:17.390190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 762/1520] 10.222.0.36 () {58 vars in 1057 bytes} [Wed Jun 22 11:43:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:43:17.397997 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bdc1d-58b5-4ce2-816c-34ca5042e9aa None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:43:17.411216 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bdc1d-58b5-4ce2-816c-34ca5042e9aa None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:43:17.416571 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0214f-087a-40ed-8166-87c5c58d4ba2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93630ed5e6dc4a5f958bc7cf1a68a26c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:43:17.417204 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0214f-087a-40ed-8166-87c5c58d4ba2 service placement] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:17.417485 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0214f-087a-40ed-8166-87c5c58d4ba2 service placement] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:17.417757 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0214f-087a-40ed-8166-87c5c58d4ba2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:17.427137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0214f-087a-40ed-8166-87c5c58d4ba2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:43:17.428751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0214f-087a-40ed-8166-87c5c58d4ba2 service placement] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:43:17.438358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 759/1521] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:43:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:43:18.732365 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adadaf5-075d-4177-9c82-b60ccb8a90b9 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:18.732930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adadaf5-075d-4177-9c82-b60ccb8a90b9 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:18.732930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adadaf5-075d-4177-9c82-b60ccb8a90b9 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:18.734347 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 22 11:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:43:18.739925 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee22da-29cf-464b-9863-f7bc56280564 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:18.740371 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee22da-29cf-464b-9863-f7bc56280564 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:18.740663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee22da-29cf-464b-9863-f7bc56280564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:18.760853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-3dee22da-29cf-464b-9863-f7bc56280564 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:43:18.762069 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-3dee22da-29cf-464b-9863-f7bc56280564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:43:18.770887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 22 11:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:43:18.929503 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adadaf5-075d-4177-9c82-b60ccb8a90b9 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:43:18.944606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adadaf5-075d-4177-9c82-b60ccb8a90b9 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:43:18.950104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b70b2b-9008-4e99-be15-59d0ab71d922 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:43:18.950743 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b70b2b-9008-4e99-be15-59d0ab71d922 service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:18.951027 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b70b2b-9008-4e99-be15-59d0ab71d922 service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:18.951304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b70b2b-9008-4e99-be15-59d0ab71d922 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:18.960521 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b70b2b-9008-4e99-be15-59d0ab71d922 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:43:18.962125 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b70b2b-9008-4e99-be15-59d0ab71d922 service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:43:18.971476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 764/1524] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:43:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:43:19.795492 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283adf78-d299-421e-ad60-b3c1dd3a0f78 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:19.796062 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283adf78-d299-421e-ad60-b3c1dd3a0f78 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:19.796062 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283adf78-d299-421e-ad60-b3c1dd3a0f78 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:19.797443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 22 11:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:43:19.802182 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161e29f-5408-4ee2-aa41-65cf5d1a73bc None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:19.802609 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161e29f-5408-4ee2-aa41-65cf5d1a73bc None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:19.802807 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161e29f-5408-4ee2-aa41-65cf5d1a73bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:19.820444 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-c161e29f-5408-4ee2-aa41-65cf5d1a73bc None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:43:19.821634 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-c161e29f-5408-4ee2-aa41-65cf5d1a73bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:43:19.829378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 22 11:43:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:43:19.992411 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283adf78-d299-421e-ad60-b3c1dd3a0f78 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:43:20.007435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283adf78-d299-421e-ad60-b3c1dd3a0f78 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:43:20.012890 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4b09c-5ece-4c92-82a2-bcd1c347bec2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:43:20.013524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4b09c-5ece-4c92-82a2-bcd1c347bec2 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:20.013812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4b09c-5ece-4c92-82a2-bcd1c347bec2 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:20.014090 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4b09c-5ece-4c92-82a2-bcd1c347bec2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:20.023226 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a4b09c-5ece-4c92-82a2-bcd1c347bec2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:43:20.024916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a4b09c-5ece-4c92-82a2-bcd1c347bec2 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:43:20.034214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 762/1527] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:43:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:43:23.911746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a6e4c-22a1-4fe2-83c8-ca8499403d11 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:23.912328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a6e4c-22a1-4fe2-83c8-ca8499403d11 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:23.912328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a6e4c-22a1-4fe2-83c8-ca8499403d11 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:23.913743 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 22 11:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:43:23.919265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50446b10-e413-466f-a831-eb40c56d7682 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:23.919729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50446b10-e413-466f-a831-eb40c56d7682 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:23.919874 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50446b10-e413-466f-a831-eb40c56d7682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:23.938278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-50446b10-e413-466f-a831-eb40c56d7682 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:43:23.939520 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-50446b10-e413-466f-a831-eb40c56d7682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:43:23.951530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 22 11:43:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:43:23.989499 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a6e4c-22a1-4fe2-83c8-ca8499403d11 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:43:23.991211 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-215a6e4c-22a1-4fe2-83c8-ca8499403d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:43:24.049437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a6e4c-22a1-4fe2-83c8-ca8499403d11 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:43:24.052966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd105f6-eaf6-4d39-8b9c-4c426fb20c1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:43:24.053612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd105f6-eaf6-4d39-8b9c-4c426fb20c1d service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:24.053891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd105f6-eaf6-4d39-8b9c-4c426fb20c1d service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:24.054166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd105f6-eaf6-4d39-8b9c-4c426fb20c1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:24.062900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd105f6-eaf6-4d39-8b9c-4c426fb20c1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:43:24.064345 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd105f6-eaf6-4d39-8b9c-4c426fb20c1d service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:43:24.069142 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:43:24.074544 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 767/1530] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:43:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:43:26.144509 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bc9d9-3ee9-44ab-a692-0911854acaf8 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:26.145061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bc9d9-3ee9-44ab-a692-0911854acaf8 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:26.145061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bc9d9-3ee9-44ab-a692-0911854acaf8 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:26.146460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 22 11:43:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:43:26.151555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53bf19-8347-40a0-85ee-f016124db168 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:26.151976 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53bf19-8347-40a0-85ee-f016124db168 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:26.152117 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53bf19-8347-40a0-85ee-f016124db168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:26.169920 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-6b53bf19-8347-40a0-85ee-f016124db168 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:43:26.171128 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-6b53bf19-8347-40a0-85ee-f016124db168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:43:26.179106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 22 11:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:43:26.215234 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bc9d9-3ee9-44ab-a692-0911854acaf8 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:43:26.225846 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589bc9d9-3ee9-44ab-a692-0911854acaf8 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:43:26.229342 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a66b23-ecd8-43ec-b60f-3f15c18d7208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:43:26.229971 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a66b23-ecd8-43ec-b60f-3f15c18d7208 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:26.230257 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a66b23-ecd8-43ec-b60f-3f15c18d7208 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:26.230532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a66b23-ecd8-43ec-b60f-3f15c18d7208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:26.239480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a66b23-ecd8-43ec-b60f-3f15c18d7208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:43:26.240899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a66b23-ecd8-43ec-b60f-3f15c18d7208 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:43:26.250331 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 765/1533] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:43:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:43:29.803176 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3e1a0-7f2c-45f0-8e80-c6e93792a0a0 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:29.803752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3e1a0-7f2c-45f0-8e80-c6e93792a0a0 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:29.803752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3e1a0-7f2c-45f0-8e80-c6e93792a0a0 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:29.805136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 22 11:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:43:29.810249 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b20ec-957e-4783-9570-e44d6aa56a02 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:29.810634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b20ec-957e-4783-9570-e44d6aa56a02 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:29.810806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b20ec-957e-4783-9570-e44d6aa56a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:29.828440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-1a5b20ec-957e-4783-9570-e44d6aa56a02 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:43:29.829641 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-1a5b20ec-957e-4783-9570-e44d6aa56a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:43:29.837358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 22 11:43:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:43:30.139928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3e1a0-7f2c-45f0-8e80-c6e93792a0a0 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:43:30.141592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-8af3e1a0-7f2c-45f0-8e80-c6e93792a0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:43:30.200084 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3e1a0-7f2c-45f0-8e80-c6e93792a0a0 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:43:30.203598 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16978899-e3a4-44be-8980-a07e91e49cea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:43:30.204233 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16978899-e3a4-44be-8980-a07e91e49cea service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:30.204512 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16978899-e3a4-44be-8980-a07e91e49cea service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:30.204785 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16978899-e3a4-44be-8980-a07e91e49cea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:30.214219 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16978899-e3a4-44be-8980-a07e91e49cea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:43:30.215665 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16978899-e3a4-44be-8980-a07e91e49cea service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:43:30.225396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 770/1536] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:43:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:43:32.277602 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502c3b1-3661-4583-86ec-69ca948df257 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:32.278193 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502c3b1-3661-4583-86ec-69ca948df257 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:32.278193 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502c3b1-3661-4583-86ec-69ca948df257 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:32.279634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 22 11:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:43:32.284922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d664f7-ccc3-4c71-86d5-21566ca129f6 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:32.285453 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d664f7-ccc3-4c71-86d5-21566ca129f6 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:32.285453 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d664f7-ccc3-4c71-86d5-21566ca129f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:32.303042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-26d664f7-ccc3-4c71-86d5-21566ca129f6 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:43:32.304271 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-26d664f7-ccc3-4c71-86d5-21566ca129f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:43:32.312036 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 22 11:43:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:43:32.618519 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502c3b1-3661-4583-86ec-69ca948df257 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:43:32.620199 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-0502c3b1-3661-4583-86ec-69ca948df257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:43:32.633384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502c3b1-3661-4583-86ec-69ca948df257 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:43:32.636812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa6a1b6-a301-4270-a0df-96f68c88d252 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:43:32.637433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6a1b6-a301-4270-a0df-96f68c88d252 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:43:32.637710 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6a1b6-a301-4270-a0df-96f68c88d252 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:43:32.637995 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6a1b6-a301-4270-a0df-96f68c88d252 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:43:32.647096 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa6a1b6-a301-4270-a0df-96f68c88d252 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:43:32.648529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa6a1b6-a301-4270-a0df-96f68c88d252 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:43:32.657627 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 768/1539] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:43:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:43:42.297982 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=32 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:43:42.653597 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=33 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:02.651065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250128d8-2404-4c74-927e-b94bcb61c18c None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:02.658975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250128d8-2404-4c74-927e-b94bcb61c18c None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:02.658975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250128d8-2404-4c74-927e-b94bcb61c18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:02.674404 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-250128d8-2404-4c74-927e-b94bcb61c18c None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:02.680289 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-250128d8-2404-4c74-927e-b94bcb61c18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:02.694703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 772/1540] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:02.706817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41088b6-e84d-4021-98fc-6d6927877c48 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:02.707467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41088b6-e84d-4021-98fc-6d6927877c48 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:02.707467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41088b6-e84d-4021-98fc-6d6927877c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:02.736989 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-f41088b6-e84d-4021-98fc-6d6927877c48 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:02.738238 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f41088b6-e84d-4021-98fc-6d6927877c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:02.774264 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 769/1541] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:02.777471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250128d8-2404-4c74-927e-b94bcb61c18c None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:02.797810 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250128d8-2404-4c74-927e-b94bcb61c18c None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:02.802779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac9ac6-aecf-42a3-8e91-d291dc63472b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:02.803498 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac9ac6-aecf-42a3-8e91-d291dc63472b admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:02.803782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac9ac6-aecf-42a3-8e91-d291dc63472b admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:02.804059 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac9ac6-aecf-42a3-8e91-d291dc63472b admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:02.804796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac9ac6-aecf-42a3-8e91-d291dc63472b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:02.811518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac9ac6-aecf-42a3-8e91-d291dc63472b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:02.813262 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac9ac6-aecf-42a3-8e91-d291dc63472b admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:02.833710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 773/1542] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:02.870049 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41088b6-e84d-4021-98fc-6d6927877c48 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:02.888724 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41088b6-e84d-4021-98fc-6d6927877c48 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:02.913210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365af19-fa09-4379-8605-e8bca75ca451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:02.914984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365af19-fa09-4379-8605-e8bca75ca451 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:02.915807 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365af19-fa09-4379-8605-e8bca75ca451 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:02.916595 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365af19-fa09-4379-8605-e8bca75ca451 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:02.918156 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365af19-fa09-4379-8605-e8bca75ca451 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:02.922513 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365af19-fa09-4379-8605-e8bca75ca451 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:03.002794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 770/1543] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:02 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:03.008042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac9ac6-aecf-42a3-8e91-d291dc63472b admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:03.034336 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac9ac6-aecf-42a3-8e91-d291dc63472b admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:03.123785 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec8f5d-2232-4b4a-90b3-2ef959c2b63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:03.124979 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec8f5d-2232-4b4a-90b3-2ef959c2b63f admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.125180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec8f5d-2232-4b4a-90b3-2ef959c2b63f admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.125864 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec8f5d-2232-4b4a-90b3-2ef959c2b63f admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.126277 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec8f5d-2232-4b4a-90b3-2ef959c2b63f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:03.134963 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec8f5d-2232-4b4a-90b3-2ef959c2b63f admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:03.179040 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ea755-cfde-432f-83c2-53acd4e70f7b None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.179488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ea755-cfde-432f-83c2-53acd4e70f7b None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.179987 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ea755-cfde-432f-83c2-53acd4e70f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.182996 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 774/1544] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:03 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:03.187512 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec8f5d-2232-4b4a-90b3-2ef959c2b63f admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:03.209237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec8f5d-2232-4b4a-90b3-2ef959c2b63f admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:03.237513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be877421-1f6a-4f20-aaa6-58ef15b3e45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:03.237513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be877421-1f6a-4f20-aaa6-58ef15b3e45a admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.237513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be877421-1f6a-4f20-aaa6-58ef15b3e45a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.237513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be877421-1f6a-4f20-aaa6-58ef15b3e45a admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.237991 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be877421-1f6a-4f20-aaa6-58ef15b3e45a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:03.248389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be877421-1f6a-4f20-aaa6-58ef15b3e45a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:03.287383 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 775/1545] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:03.299190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-3c5ea755-cfde-432f-83c2-53acd4e70f7b None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:03.299470 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b77190-cb7c-4d58-8b70-433698c0db9e None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.299470 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b77190-cb7c-4d58-8b70-433698c0db9e None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.299470 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b77190-cb7c-4d58-8b70-433698c0db9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.299709 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-3c5ea755-cfde-432f-83c2-53acd4e70f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:03.366924 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 771/1546] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:45:03.383379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ea755-cfde-432f-83c2-53acd4e70f7b None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:03.411387 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d6b77190-cb7c-4d58-8b70-433698c0db9e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:03.411387 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d6b77190-cb7c-4d58-8b70-433698c0db9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:03.454816 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 776/1547] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:03.456206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66971b15-532e-4de0-805c-a43eab03edbb None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.456502 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66971b15-532e-4de0-805c-a43eab03edbb None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.456783 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66971b15-532e-4de0-805c-a43eab03edbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.458745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ea755-cfde-432f-83c2-53acd4e70f7b None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:03.487506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb2a45-94c4-4fc8-891b-40ea618c7017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:03.488672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb2a45-94c4-4fc8-891b-40ea618c7017 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.489162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb2a45-94c4-4fc8-891b-40ea618c7017 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.489634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb2a45-94c4-4fc8-891b-40ea618c7017 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.497067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb2a45-94c4-4fc8-891b-40ea618c7017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:03.503962 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb2a45-94c4-4fc8-891b-40ea618c7017 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:03.508618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-66971b15-532e-4de0-805c-a43eab03edbb None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:03.509854 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-66971b15-532e-4de0-805c-a43eab03edbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:03.541134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 777/1548] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:03.542471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ae2c1-c059-4af5-93cf-c1809344c64e None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.542763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ae2c1-c059-4af5-93cf-c1809344c64e None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.543040 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ae2c1-c059-4af5-93cf-c1809344c64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.547498 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 772/1549] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:03.555579 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1ed50-ac14-44f2-992e-5ddd7a74cd87 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.557125 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1ed50-ac14-44f2-992e-5ddd7a74cd87 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.557622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1ed50-ac14-44f2-992e-5ddd7a74cd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.612458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-919ae2c1-c059-4af5-93cf-c1809344c64e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:03.615375 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c0a1ed50-ac14-44f2-992e-5ddd7a74cd87 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:03.616567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-919ae2c1-c059-4af5-93cf-c1809344c64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:03.620124 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c0a1ed50-ac14-44f2-992e-5ddd7a74cd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:03.628323 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 778/1550] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:03.629808 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f6380-4342-4cd0-a5fc-12cdf0399edd None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.630506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f6380-4342-4cd0-a5fc-12cdf0399edd None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.630506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f6380-4342-4cd0-a5fc-12cdf0399edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.638510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 773/1551] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:03.639787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1ed50-ac14-44f2-992e-5ddd7a74cd87 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:03.655659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1ed50-ac14-44f2-992e-5ddd7a74cd87 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:03.663589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ae969-ae97-4cab-8ad3-5e494d12f2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:03.665892 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ae969-ae97-4cab-8ad3-5e494d12f2a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.667056 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-e20f6380-4342-4cd0-a5fc-12cdf0399edd None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:03.667967 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ae969-ae97-4cab-8ad3-5e494d12f2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.668475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ae969-ae97-4cab-8ad3-5e494d12f2a5 admin admin] PATH_INFO: `/v3/projects/35df61dc9f3544a4ad9fa3be324d1e8b/users/3b8a3e35ce254868bf3825cf60139979/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.670779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e20f6380-4342-4cd0-a5fc-12cdf0399edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:03.693783 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 779/1552] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:03.695005 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f6380-4342-4cd0-a5fc-12cdf0399edd None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:03.705142 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ae969-ae97-4cab-8ad3-5e494d12f2a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35df61dc9f3544a4ad9fa3be324d1e8b, user_id=3b8a3e35ce254868bf3825cf60139979, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:03.708172 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ae969-ae97-4cab-8ad3-5e494d12f2a5 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:03.720699 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f6380-4342-4cd0-a5fc-12cdf0399edd None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:03.726643 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 774/1553] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:03 2022] PUT /identity/v3/projects/35df61dc9f3544a4ad9fa3be324d1e8b/users/3b8a3e35ce254868bf3825cf60139979/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:03.732765 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16303ad3-c168-4d1f-a68e-d9aa3edde83d None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.733328 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16303ad3-c168-4d1f-a68e-d9aa3edde83d None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.734384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16303ad3-c168-4d1f-a68e-d9aa3edde83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.756818 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-16303ad3-c168-4d1f-a68e-d9aa3edde83d None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:03.758306 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-16303ad3-c168-4d1f-a68e-d9aa3edde83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:03.814365 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6a42c-3778-4445-a634-03e360f3ba06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:03.815031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6a42c-3778-4445-a634-03e360f3ba06 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.815308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6a42c-3778-4445-a634-03e360f3ba06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.815608 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6a42c-3778-4445-a634-03e360f3ba06 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.816329 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6a42c-3778-4445-a634-03e360f3ba06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:03.816633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6a42c-3778-4445-a634-03e360f3ba06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:03.818229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee6a42c-3778-4445-a634-03e360f3ba06 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:03.829952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 780/1554] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:03.830890 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6a42c-3778-4445-a634-03e360f3ba06 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:03.839614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 775/1555] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:03.839614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739cfa93-7f25-429c-850e-52c03081db6d None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.839614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739cfa93-7f25-429c-850e-52c03081db6d None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.839614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739cfa93-7f25-429c-850e-52c03081db6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.842555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6a42c-3778-4445-a634-03e360f3ba06 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:03.847987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f08dc4-761d-4d6c-92ff-9d88ef95a6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:03.848654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f08dc4-761d-4d6c-92ff-9d88ef95a6f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.848936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f08dc4-761d-4d6c-92ff-9d88ef95a6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.849213 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f08dc4-761d-4d6c-92ff-9d88ef95a6f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.850006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f08dc4-761d-4d6c-92ff-9d88ef95a6f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:03.850258 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f08dc4-761d-4d6c-92ff-9d88ef95a6f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:03.851919 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f08dc4-761d-4d6c-92ff-9d88ef95a6f4 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:03.856639 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-739cfa93-7f25-429c-850e-52c03081db6d None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:03.857862 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-739cfa93-7f25-429c-850e-52c03081db6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:03.865238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 781/1556] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:03.865948 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be38e0f-305e-4788-a40c-c2dd516f0399 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.866236 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be38e0f-305e-4788-a40c-c2dd516f0399 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.866634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 776/1557] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:03.866767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be38e0f-305e-4788-a40c-c2dd516f0399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.868825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cd191-9526-4347-8405-b25fe2433376 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.869113 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cd191-9526-4347-8405-b25fe2433376 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.869398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cd191-9526-4347-8405-b25fe2433376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.887562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-d00cd191-9526-4347-8405-b25fe2433376 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:03.888704 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-5be38e0f-305e-4788-a40c-c2dd516f0399 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:03.888883 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d00cd191-9526-4347-8405-b25fe2433376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:03.890210 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5be38e0f-305e-4788-a40c-c2dd516f0399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:03.897157 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 777/1558] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:03.898399 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d49a1-6529-4a96-865f-451cb15d8ed9 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.898683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d49a1-6529-4a96-865f-451cb15d8ed9 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.898956 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d49a1-6529-4a96-865f-451cb15d8ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.899441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 782/1559] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:03.900385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056c21e-9a45-4e27-b62a-a42776c4c97c None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.900674 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056c21e-9a45-4e27-b62a-a42776c4c97c None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.900950 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056c21e-9a45-4e27-b62a-a42776c4c97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.914659 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:03.920300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-0c2d49a1-6529-4a96-865f-451cb15d8ed9 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:03.921459 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-b056c21e-9a45-4e27-b62a-a42776c4c97c None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:03.921639 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-0c2d49a1-6529-4a96-865f-451cb15d8ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:03.922823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b056c21e-9a45-4e27-b62a-a42776c4c97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:03.931144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 778/1560] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:03.933390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 783/1561] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:03.942351 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d49a1-6529-4a96-865f-451cb15d8ed9 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:03.943004 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab506d9d-187c-41c0-85f5-b638ff882bb2 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.943004 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab506d9d-187c-41c0-85f5-b638ff882bb2 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.943004 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab506d9d-187c-41c0-85f5-b638ff882bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.950133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d49a1-6529-4a96-865f-451cb15d8ed9 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:03.954857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e041c2-2912-45c4-a086-7dc2a2354cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:03.955611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e041c2-2912-45c4-a086-7dc2a2354cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.955839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e041c2-2912-45c4-a086-7dc2a2354cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.956115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e041c2-2912-45c4-a086-7dc2a2354cf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.956808 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e041c2-2912-45c4-a086-7dc2a2354cf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:03.958724 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e041c2-2912-45c4-a086-7dc2a2354cf4 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:03.960553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-ab506d9d-187c-41c0-85f5-b638ff882bb2 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:03.961732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ab506d9d-187c-41c0-85f5-b638ff882bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:03.969605 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 784/1562] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:03.970942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab506d9d-187c-41c0-85f5-b638ff882bb2 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:03.981091 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab506d9d-187c-41c0-85f5-b638ff882bb2 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:03.985726 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 779/1563] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:03 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:03.987216 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e041c2-2912-45c4-a086-7dc2a2354cf4 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:03.987552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b6809-6b05-4a34-8af2-ee7109af75e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:03.988225 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b6809-6b05-4a34-8af2-ee7109af75e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:03.988515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b6809-6b05-4a34-8af2-ee7109af75e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:03.988793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b6809-6b05-4a34-8af2-ee7109af75e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:03.989519 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b6809-6b05-4a34-8af2-ee7109af75e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:03.989826 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b6809-6b05-4a34-8af2-ee7109af75e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:03.991476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b6809-6b05-4a34-8af2-ee7109af75e7 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.010685 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 785/1564] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:04.010685 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b6809-6b05-4a34-8af2-ee7109af75e7 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.016961 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b6809-6b05-4a34-8af2-ee7109af75e7 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.044486 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:04.063240 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e041c2-2912-45c4-a086-7dc2a2354cf4 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.069174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15419e0e-58a7-495a-b628-7dc7eca551fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.070012 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15419e0e-58a7-495a-b628-7dc7eca551fd admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.070304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15419e0e-58a7-495a-b628-7dc7eca551fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.070589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15419e0e-58a7-495a-b628-7dc7eca551fd admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.071300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15419e0e-58a7-495a-b628-7dc7eca551fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.073231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15419e0e-58a7-495a-b628-7dc7eca551fd admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.079114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcb874-9212-4a7a-89d7-10fe1cc2b597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.079898 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcb874-9212-4a7a-89d7-10fe1cc2b597 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.080739 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcb874-9212-4a7a-89d7-10fe1cc2b597 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.081040 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcb874-9212-4a7a-89d7-10fe1cc2b597 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.081831 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcb874-9212-4a7a-89d7-10fe1cc2b597 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:04.082119 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcb874-9212-4a7a-89d7-10fe1cc2b597 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.083980 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcb874-9212-4a7a-89d7-10fe1cc2b597 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.097540 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 786/1565] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:04.098896 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48b007-1d55-4e6f-b926-34bdc734aa9e None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.099185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48b007-1d55-4e6f-b926-34bdc734aa9e None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.099489 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48b007-1d55-4e6f-b926-34bdc734aa9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.109321 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 780/1566] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:03 2022] POST /identity/v3/projects => generated 391 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:04.110235 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15419e0e-58a7-495a-b628-7dc7eca551fd admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.116765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-ab48b007-1d55-4e6f-b926-34bdc734aa9e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:04.117991 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ab48b007-1d55-4e6f-b926-34bdc734aa9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:04.165716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15419e0e-58a7-495a-b628-7dc7eca551fd admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.169479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72365fc-f17d-4bf9-801e-04756b8c1b13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.170124 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72365fc-f17d-4bf9-801e-04756b8c1b13 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.170404 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72365fc-f17d-4bf9-801e-04756b8c1b13 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.170790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72365fc-f17d-4bf9-801e-04756b8c1b13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.178843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 787/1567] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:04.180221 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c77ebc-c735-4435-8535-7c3a964a7250 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.180515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c77ebc-c735-4435-8535-7c3a964a7250 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.180795 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c77ebc-c735-4435-8535-7c3a964a7250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.199221 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d5c77ebc-c735-4435-8535-7c3a964a7250 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:04.200453 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d5c77ebc-c735-4435-8535-7c3a964a7250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:04.208582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 788/1568] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:04.209326 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c77ebc-c735-4435-8535-7c3a964a7250 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.218469 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c77ebc-c735-4435-8535-7c3a964a7250 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.223360 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b6ba7-6b96-4de6-a06d-baf8fa2844ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.224023 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b6ba7-6b96-4de6-a06d-baf8fa2844ff admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.224307 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b6ba7-6b96-4de6-a06d-baf8fa2844ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.224584 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b6ba7-6b96-4de6-a06d-baf8fa2844ff admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.225296 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b6ba7-6b96-4de6-a06d-baf8fa2844ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:04.225602 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b6ba7-6b96-4de6-a06d-baf8fa2844ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.227227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b6ba7-6b96-4de6-a06d-baf8fa2844ff admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.239671 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72365fc-f17d-4bf9-801e-04756b8c1b13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.241115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72365fc-f17d-4bf9-801e-04756b8c1b13 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.241402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 789/1569] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:04.242227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b6ba7-6b96-4de6-a06d-baf8fa2844ff admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.252454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b6ba7-6b96-4de6-a06d-baf8fa2844ff admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.257389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63941aa-169a-4b38-a8bc-8d9b29e5bbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.258050 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63941aa-169a-4b38-a8bc-8d9b29e5bbe1 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.258336 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63941aa-169a-4b38-a8bc-8d9b29e5bbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.258563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 781/1570] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:04.258670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63941aa-169a-4b38-a8bc-8d9b29e5bbe1 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.259433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63941aa-169a-4b38-a8bc-8d9b29e5bbe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.259644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72365fc-f17d-4bf9-801e-04756b8c1b13 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.261433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63941aa-169a-4b38-a8bc-8d9b29e5bbe1 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.269834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72365fc-f17d-4bf9-801e-04756b8c1b13 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.275581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a9c0e-1da2-4009-b254-fa115812e205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.276476 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a9c0e-1da2-4009-b254-fa115812e205 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.277045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a9c0e-1da2-4009-b254-fa115812e205 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.277653 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a9c0e-1da2-4009-b254-fa115812e205 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.278502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a9c0e-1da2-4009-b254-fa115812e205 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.280469 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a9c0e-1da2-4009-b254-fa115812e205 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.286445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 790/1571] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:04 2022] POST /identity/v3/users => generated 287 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:04.286445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63941aa-169a-4b38-a8bc-8d9b29e5bbe1 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.294730 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63941aa-169a-4b38-a8bc-8d9b29e5bbe1 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.300498 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d85cc9-c7d2-4fd2-96a8-365419a72470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.301202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d85cc9-c7d2-4fd2-96a8-365419a72470 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.301363 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d85cc9-c7d2-4fd2-96a8-365419a72470 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.301683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d85cc9-c7d2-4fd2-96a8-365419a72470 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.302330 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d85cc9-c7d2-4fd2-96a8-365419a72470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.303485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: WARNING py.warnings [None req-a1d85cc9-c7d2-4fd2-96a8-365419a72470 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 22 11:45:04.303485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: warnings.warn(msg) Jun 22 11:45:04.303485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: Jun 22 11:45:04.304813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d85cc9-c7d2-4fd2-96a8-365419a72470 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.315461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 782/1572] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:04 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:04.316262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a9c0e-1da2-4009-b254-fa115812e205 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.336509 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 791/1573] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:04 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:04.338457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d85cc9-c7d2-4fd2-96a8-365419a72470 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.361950 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:04.381984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:04.388552 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a9c0e-1da2-4009-b254-fa115812e205 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.417510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d85cc9-c7d2-4fd2-96a8-365419a72470 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.422842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0845b08-0690-4145-a17d-a5f5c99aae60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.424179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0845b08-0690-4145-a17d-a5f5c99aae60 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.424632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0845b08-0690-4145-a17d-a5f5c99aae60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.425109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0845b08-0690-4145-a17d-a5f5c99aae60 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.426219 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0845b08-0690-4145-a17d-a5f5c99aae60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:04.426574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0845b08-0690-4145-a17d-a5f5c99aae60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.428957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0845b08-0690-4145-a17d-a5f5c99aae60 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.447495 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 792/1574] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:04.447495 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0845b08-0690-4145-a17d-a5f5c99aae60 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.457790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65885387-42b9-4e13-a19e-284399e797c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.458979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65885387-42b9-4e13-a19e-284399e797c5 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.459211 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65885387-42b9-4e13-a19e-284399e797c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.459979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65885387-42b9-4e13-a19e-284399e797c5 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.460229 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65885387-42b9-4e13-a19e-284399e797c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.462737 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0845b08-0690-4145-a17d-a5f5c99aae60 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.464271 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: WARNING py.warnings [None req-65885387-42b9-4e13-a19e-284399e797c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 22 11:45:04.464271 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: warnings.warn(msg) Jun 22 11:45:04.464271 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: Jun 22 11:45:04.465634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65885387-42b9-4e13-a19e-284399e797c5 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.473797 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ecd9e-61ad-43f5-a585-bfcf18293ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.474845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ecd9e-61ad-43f5-a585-bfcf18293ccf admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.475370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ecd9e-61ad-43f5-a585-bfcf18293ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.476069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ecd9e-61ad-43f5-a585-bfcf18293ccf admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.477199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ecd9e-61ad-43f5-a585-bfcf18293ccf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:04.477835 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ecd9e-61ad-43f5-a585-bfcf18293ccf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.479869 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ecd9e-61ad-43f5-a585-bfcf18293ccf admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.494719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 793/1575] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:04.498162 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ecd9e-61ad-43f5-a585-bfcf18293ccf admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.498722 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 783/1576] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:04 2022] POST /identity/v3/users => generated 296 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:04.499985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65885387-42b9-4e13-a19e-284399e797c5 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.511217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ecd9e-61ad-43f5-a585-bfcf18293ccf admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.512408 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65885387-42b9-4e13-a19e-284399e797c5 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.516123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3683e91-0a09-4a1e-9d0c-74249c462797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.516876 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3683e91-0a09-4a1e-9d0c-74249c462797 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.517597 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30655907-4c3b-4223-8278-2c201d5c7499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.517736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3683e91-0a09-4a1e-9d0c-74249c462797 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.517964 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3683e91-0a09-4a1e-9d0c-74249c462797 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.518544 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30655907-4c3b-4223-8278-2c201d5c7499 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.518671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3683e91-0a09-4a1e-9d0c-74249c462797 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.518833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30655907-4c3b-4223-8278-2c201d5c7499 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.519192 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30655907-4c3b-4223-8278-2c201d5c7499 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.520266 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30655907-4c3b-4223-8278-2c201d5c7499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.520603 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3683e91-0a09-4a1e-9d0c-74249c462797 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.522154 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30655907-4c3b-4223-8278-2c201d5c7499 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.536632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 784/1577] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:04.536632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30655907-4c3b-4223-8278-2c201d5c7499 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.549382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30655907-4c3b-4223-8278-2c201d5c7499 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.553462 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 794/1578] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:04 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:04.554334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b396ac-655f-4a11-bdf3-ac74bd2e69a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.555125 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b396ac-655f-4a11-bdf3-ac74bd2e69a2 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.555503 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b396ac-655f-4a11-bdf3-ac74bd2e69a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.555849 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b396ac-655f-4a11-bdf3-ac74bd2e69a2 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.556607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b396ac-655f-4a11-bdf3-ac74bd2e69a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.557336 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3683e91-0a09-4a1e-9d0c-74249c462797 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.558653 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b396ac-655f-4a11-bdf3-ac74bd2e69a2 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.582443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 785/1579] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:04 2022] POST /identity/v3/users => generated 283 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:04.583255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b396ac-655f-4a11-bdf3-ac74bd2e69a2 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.595911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b396ac-655f-4a11-bdf3-ac74bd2e69a2 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.625549 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3683e91-0a09-4a1e-9d0c-74249c462797 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.630897 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca234277-f846-46d8-b3b6-1f97e974ac82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.631713 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca234277-f846-46d8-b3b6-1f97e974ac82 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.632064 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca234277-f846-46d8-b3b6-1f97e974ac82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.632508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca234277-f846-46d8-b3b6-1f97e974ac82 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.633211 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca234277-f846-46d8-b3b6-1f97e974ac82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.635703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca234277-f846-46d8-b3b6-1f97e974ac82 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.663408 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 795/1580] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:04 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:04.663408 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca234277-f846-46d8-b3b6-1f97e974ac82 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.672315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc01e4-9ccd-4a9b-8cc7-acd784526395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.673202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca234277-f846-46d8-b3b6-1f97e974ac82 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.674867 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc01e4-9ccd-4a9b-8cc7-acd784526395 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.675161 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc01e4-9ccd-4a9b-8cc7-acd784526395 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.677053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc01e4-9ccd-4a9b-8cc7-acd784526395 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.678098 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc01e4-9ccd-4a9b-8cc7-acd784526395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.678940 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed85b84-a0ee-490f-8169-194375f50121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.679682 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed85b84-a0ee-490f-8169-194375f50121 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.679965 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed85b84-a0ee-490f-8169-194375f50121 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.680249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed85b84-a0ee-490f-8169-194375f50121 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.680970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed85b84-a0ee-490f-8169-194375f50121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.682339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc01e4-9ccd-4a9b-8cc7-acd784526395 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.682555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed85b84-a0ee-490f-8169-194375f50121 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.694961 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 796/1581] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:04.695823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed85b84-a0ee-490f-8169-194375f50121 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.705855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed85b84-a0ee-490f-8169-194375f50121 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.710811 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1416b6-81d0-4a97-b1d9-e26b8ae5cd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.711430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1416b6-81d0-4a97-b1d9-e26b8ae5cd97 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.711716 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1416b6-81d0-4a97-b1d9-e26b8ae5cd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.711990 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1416b6-81d0-4a97-b1d9-e26b8ae5cd97 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.712675 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1416b6-81d0-4a97-b1d9-e26b8ae5cd97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.714572 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1416b6-81d0-4a97-b1d9-e26b8ae5cd97 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.716652 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 786/1582] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:04 2022] POST /identity/v3/projects => generated 365 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:04.721086 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc01e4-9ccd-4a9b-8cc7-acd784526395 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.744366 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 797/1583] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:04 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:04.745247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1416b6-81d0-4a97-b1d9-e26b8ae5cd97 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.781274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:04.786862 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:04.805718 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc01e4-9ccd-4a9b-8cc7-acd784526395 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.824340 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1416b6-81d0-4a97-b1d9-e26b8ae5cd97 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.829717 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6b8a2-5994-4480-9bbb-d9025c825865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.830393 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6b8a2-5994-4480-9bbb-d9025c825865 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.830941 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6b8a2-5994-4480-9bbb-d9025c825865 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.832032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6b8a2-5994-4480-9bbb-d9025c825865 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.832032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6b8a2-5994-4480-9bbb-d9025c825865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.834181 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6b8a2-5994-4480-9bbb-d9025c825865 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.860765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 798/1584] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:04 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:04.860765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6b8a2-5994-4480-9bbb-d9025c825865 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.871252 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6b8a2-5994-4480-9bbb-d9025c825865 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.876094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ccf8d-3290-4087-9a5b-a018c42dafb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.876544 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebdaa1-4ca0-4ab7-a797-374c9da4dddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.876879 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ccf8d-3290-4087-9a5b-a018c42dafb3 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.877093 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ccf8d-3290-4087-9a5b-a018c42dafb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.877291 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebdaa1-4ca0-4ab7-a797-374c9da4dddf admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.877537 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ccf8d-3290-4087-9a5b-a018c42dafb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.877653 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebdaa1-4ca0-4ab7-a797-374c9da4dddf admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.877842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebdaa1-4ca0-4ab7-a797-374c9da4dddf admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.878280 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ccf8d-3290-4087-9a5b-a018c42dafb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.878765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebdaa1-4ca0-4ab7-a797-374c9da4dddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.880173 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ccf8d-3290-4087-9a5b-a018c42dafb3 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.881025 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebdaa1-4ca0-4ab7-a797-374c9da4dddf admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.892596 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 787/1585] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:04.893601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ccf8d-3290-4087-9a5b-a018c42dafb3 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.895765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 799/1586] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:04.896666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebdaa1-4ca0-4ab7-a797-374c9da4dddf admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.904811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ccf8d-3290-4087-9a5b-a018c42dafb3 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.906758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebdaa1-4ca0-4ab7-a797-374c9da4dddf admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.910187 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9493b07-2ca1-43f9-bfe7-41a3ee039872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.911506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9493b07-2ca1-43f9-bfe7-41a3ee039872 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.911793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9493b07-2ca1-43f9-bfe7-41a3ee039872 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.912073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9493b07-2ca1-43f9-bfe7-41a3ee039872 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.912804 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9493b07-2ca1-43f9-bfe7-41a3ee039872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.914032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023ac2c-d8d8-4ef8-894e-f3c4d08f50fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.914596 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9493b07-2ca1-43f9-bfe7-41a3ee039872 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.915441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023ac2c-d8d8-4ef8-894e-f3c4d08f50fe admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.915962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023ac2c-d8d8-4ef8-894e-f3c4d08f50fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.916249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023ac2c-d8d8-4ef8-894e-f3c4d08f50fe admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.917010 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023ac2c-d8d8-4ef8-894e-f3c4d08f50fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.920711 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023ac2c-d8d8-4ef8-894e-f3c4d08f50fe admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.929876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 788/1587] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:04.930610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9493b07-2ca1-43f9-bfe7-41a3ee039872 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.935973 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 800/1588] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:04.937175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023ac2c-d8d8-4ef8-894e-f3c4d08f50fe admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.943722 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9493b07-2ca1-43f9-bfe7-41a3ee039872 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.949664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f80d9-07b6-4664-a11f-67f31e109555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.950880 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f80d9-07b6-4664-a11f-67f31e109555 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.951177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f80d9-07b6-4664-a11f-67f31e109555 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.951578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f80d9-07b6-4664-a11f-67f31e109555 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.952517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f80d9-07b6-4664-a11f-67f31e109555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.954349 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f80d9-07b6-4664-a11f-67f31e109555 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.955317 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023ac2c-d8d8-4ef8-894e-f3c4d08f50fe admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:04.976826 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ef7d4-3841-4741-9574-ae0f3cd03634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:04.977831 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ef7d4-3841-4741-9574-ae0f3cd03634 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:04.978035 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ef7d4-3841-4741-9574-ae0f3cd03634 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:04.979155 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ef7d4-3841-4741-9574-ae0f3cd03634 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:04.980035 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ef7d4-3841-4741-9574-ae0f3cd03634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:04.981071 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 789/1589] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:04 2022] POST /identity/v3/users => generated 283 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:04.982162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f80d9-07b6-4664-a11f-67f31e109555 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:04.986452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ef7d4-3841-4741-9574-ae0f3cd03634 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:04.993608 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f80d9-07b6-4664-a11f-67f31e109555 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:05.001059 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7174a8-5662-48d4-bdff-4f3613f0a409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:05.002320 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7174a8-5662-48d4-bdff-4f3613f0a409 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:05.002620 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7174a8-5662-48d4-bdff-4f3613f0a409 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:05.002895 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7174a8-5662-48d4-bdff-4f3613f0a409 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:05.004237 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7174a8-5662-48d4-bdff-4f3613f0a409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:05.005866 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7174a8-5662-48d4-bdff-4f3613f0a409 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:05.015840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 801/1590] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:04 2022] POST /identity/v3/users => generated 286 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:05.017654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ef7d4-3841-4741-9574-ae0f3cd03634 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:05.023481 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 790/1591] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:05.024515 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7174a8-5662-48d4-bdff-4f3613f0a409 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:05.030082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ef7d4-3841-4741-9574-ae0f3cd03634 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:05.037814 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a20e6d-859d-4a24-bb91-df74bfb81400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:05.039078 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a20e6d-859d-4a24-bb91-df74bfb81400 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:05.039479 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a20e6d-859d-4a24-bb91-df74bfb81400 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:05.039773 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a20e6d-859d-4a24-bb91-df74bfb81400 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:05.040637 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a20e6d-859d-4a24-bb91-df74bfb81400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:05.042535 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a20e6d-859d-4a24-bb91-df74bfb81400 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:05.054259 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7174a8-5662-48d4-bdff-4f3613f0a409 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:05.056180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 802/1592] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:05.060232 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a20e6d-859d-4a24-bb91-df74bfb81400 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:05.065901 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd277d3-7026-474f-9a02-885a513601de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:05.068362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd277d3-7026-474f-9a02-885a513601de admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:05.069389 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd277d3-7026-474f-9a02-885a513601de admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:05.070462 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd277d3-7026-474f-9a02-885a513601de admin admin] PATH_INFO: `/v3/projects/2889a1aa2fa64d6c8d83a9893cdd7b21/users/54b2ffa75d014f4691421782afb8664f/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:05.071336 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a20e6d-859d-4a24-bb91-df74bfb81400 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:05.076188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2598d52-6025-4509-a9a5-a75ea18a715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:05.076857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2598d52-6025-4509-a9a5-a75ea18a715a admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:05.077143 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2598d52-6025-4509-a9a5-a75ea18a715a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:05.077578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2598d52-6025-4509-a9a5-a75ea18a715a admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:05.078868 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2598d52-6025-4509-a9a5-a75ea18a715a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:05.080696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2598d52-6025-4509-a9a5-a75ea18a715a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:05.096316 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 803/1593] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:05.097310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2598d52-6025-4509-a9a5-a75ea18a715a admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:05.097426 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd277d3-7026-474f-9a02-885a513601de admin admin] RBAC: Authorizing `identity:create_grant(project_id=2889a1aa2fa64d6c8d83a9893cdd7b21, user_id=54b2ffa75d014f4691421782afb8664f, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:05.100169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd277d3-7026-474f-9a02-885a513601de admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:05.108384 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2598d52-6025-4509-a9a5-a75ea18a715a admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:05.114901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b7efe-cdec-4948-b724-7e62bb552bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:05.115855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b7efe-cdec-4948-b724-7e62bb552bbf admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:05.116207 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b7efe-cdec-4948-b724-7e62bb552bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:05.116482 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b7efe-cdec-4948-b724-7e62bb552bbf admin admin] PATH_INFO: `/v3/projects/d20536d2fa444e97b7184c10e52d5cc5/users/27cc3d38926f470c91eeb6aecfc86a48/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:05.117235 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 791/1594] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:05 2022] PUT /identity/v3/projects/2889a1aa2fa64d6c8d83a9893cdd7b21/users/54b2ffa75d014f4691421782afb8664f/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:05.120344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd277d3-7026-474f-9a02-885a513601de admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:05.134635 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b7efe-cdec-4948-b724-7e62bb552bbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d20536d2fa444e97b7184c10e52d5cc5, user_id=27cc3d38926f470c91eeb6aecfc86a48, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:05.135891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: WARNING py.warnings [None req-087b7efe-cdec-4948-b724-7e62bb552bbf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 22 11:45:05.135891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: warnings.warn(msg) Jun 22 11:45:05.135891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: Jun 22 11:45:05.139088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b7efe-cdec-4948-b724-7e62bb552bbf admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:05.160636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 804/1595] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:05 2022] PUT /identity/v3/projects/d20536d2fa444e97b7184c10e52d5cc5/users/27cc3d38926f470c91eeb6aecfc86a48/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:05.162823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b7efe-cdec-4948-b724-7e62bb552bbf admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:05.262487 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b7efe-cdec-4948-b724-7e62bb552bbf admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:05.269708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ae937-465d-413b-81ae-489724927369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:05.270803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ae937-465d-413b-81ae-489724927369 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:05.271380 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ae937-465d-413b-81ae-489724927369 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:05.271713 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ae937-465d-413b-81ae-489724927369 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:05.272504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ae937-465d-413b-81ae-489724927369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:05.275309 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ae937-465d-413b-81ae-489724927369 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:05.295911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 805/1596] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:05.297737 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ae937-465d-413b-81ae-489724927369 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:05.313544 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ae937-465d-413b-81ae-489724927369 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:05.320790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d9f1b-b272-4b3a-ba33-b64d76e9d24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:05.322368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d9f1b-b272-4b3a-ba33-b64d76e9d24b admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:05.322368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d9f1b-b272-4b3a-ba33-b64d76e9d24b admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:05.322800 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d9f1b-b272-4b3a-ba33-b64d76e9d24b admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:05.324178 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d9f1b-b272-4b3a-ba33-b64d76e9d24b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:05.327049 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d9f1b-b272-4b3a-ba33-b64d76e9d24b admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:05.345106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 806/1597] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:05.346540 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d9f1b-b272-4b3a-ba33-b64d76e9d24b admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:05.356584 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d9f1b-b272-4b3a-ba33-b64d76e9d24b admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:05.359190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd277d3-7026-474f-9a02-885a513601de admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:05.362361 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d7d16-fc4a-47b7-b8bb-c0fe118f0952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:05.363407 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d7d16-fc4a-47b7-b8bb-c0fe118f0952 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:05.363796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d7d16-fc4a-47b7-b8bb-c0fe118f0952 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:05.364170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d7d16-fc4a-47b7-b8bb-c0fe118f0952 admin admin] PATH_INFO: `/v3/projects/57a9df2fc2d44e99b49cbb97ba77ba74/users/6584df0ad24c4fbabc162bb9ffc9e3f4/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:05.366227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c91940-8dbd-4a11-b016-d3432502bab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:05.367098 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c91940-8dbd-4a11-b016-d3432502bab2 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:05.367582 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c91940-8dbd-4a11-b016-d3432502bab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:05.368007 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c91940-8dbd-4a11-b016-d3432502bab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:05.368889 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c91940-8dbd-4a11-b016-d3432502bab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:05.370647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c91940-8dbd-4a11-b016-d3432502bab2 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:05.382115 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d7d16-fc4a-47b7-b8bb-c0fe118f0952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57a9df2fc2d44e99b49cbb97ba77ba74, user_id=6584df0ad24c4fbabc162bb9ffc9e3f4, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:05.385391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 792/1598] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:05.386672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c91940-8dbd-4a11-b016-d3432502bab2 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:05.388684 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d7d16-fc4a-47b7-b8bb-c0fe118f0952 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:05.394868 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:05.401862 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c91940-8dbd-4a11-b016-d3432502bab2 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:05.411959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 807/1599] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:05 2022] PUT /identity/v3/projects/57a9df2fc2d44e99b49cbb97ba77ba74/users/6584df0ad24c4fbabc162bb9ffc9e3f4/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:05.423971 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fe272-7852-4b7b-83c3-dde45ee8b2a3 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:05.424377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fe272-7852-4b7b-83c3-dde45ee8b2a3 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:05.424792 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fe272-7852-4b7b-83c3-dde45ee8b2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:05.429637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:05.468756 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-337fe272-7852-4b7b-83c3-dde45ee8b2a3 None None] MFA Rules not processed for user `54b2ffa75d014f4691421782afb8664f`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:05.470397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-337fe272-7852-4b7b-83c3-dde45ee8b2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:05.487753 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ecc866-8e62-48fe-bbce-97deffebab6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:05.489216 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ecc866-8e62-48fe-bbce-97deffebab6e admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:05.489727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ecc866-8e62-48fe-bbce-97deffebab6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:05.490499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ecc866-8e62-48fe-bbce-97deffebab6e admin admin] PATH_INFO: `/v3/projects/57c834b746fc4a11aa0226019923d4d0/users/60c13dd599844219a310f5eb222ef41f/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:05.516048 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ecc866-8e62-48fe-bbce-97deffebab6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=57c834b746fc4a11aa0226019923d4d0, user_id=60c13dd599844219a310f5eb222ef41f, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:05.519914 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ecc866-8e62-48fe-bbce-97deffebab6e admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:05.553533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 793/1600] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:05 2022] PUT /identity/v3/projects/57c834b746fc4a11aa0226019923d4d0/users/60c13dd599844219a310f5eb222ef41f/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:05.555615 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617f48a-79bb-4f24-bb95-93db4ccb4805 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:05.555993 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617f48a-79bb-4f24-bb95-93db4ccb4805 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:05.556366 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617f48a-79bb-4f24-bb95-93db4ccb4805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:05.558428 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 808/1601] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:05.559871 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fe272-7852-4b7b-83c3-dde45ee8b2a3 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:05.574723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fe272-7852-4b7b-83c3-dde45ee8b2a3 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:05.603126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-5617f48a-79bb-4f24-bb95-93db4ccb4805 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:05.605007 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-5617f48a-79bb-4f24-bb95-93db4ccb4805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:05.668954 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830a01c-71f7-4653-b504-efc8a9955d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:05.670138 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830a01c-71f7-4653-b504-efc8a9955d3c admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:05.670627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830a01c-71f7-4653-b504-efc8a9955d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:05.671128 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830a01c-71f7-4653-b504-efc8a9955d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:05.672439 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830a01c-71f7-4653-b504-efc8a9955d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:05.675542 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830a01c-71f7-4653-b504-efc8a9955d3c admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:05.690560 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 809/1602] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:05.690560 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830a01c-71f7-4653-b504-efc8a9955d3c admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:05.701456 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830a01c-71f7-4653-b504-efc8a9955d3c admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:05.707623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731cd35b-5695-42be-b5b6-eaa73900cd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:05.707955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 794/1603] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:05.708715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cd35b-5695-42be-b5b6-eaa73900cd5a admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:05.709064 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cd35b-5695-42be-b5b6-eaa73900cd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:05.709405 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cd35b-5695-42be-b5b6-eaa73900cd5a admin admin] PATH_INFO: `/v3/projects/878b71f291ec4107a09e3109b6acc40b/users/4d7c1adbb2ad456eaf5793a15d6fcb4b/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:05.710115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617f48a-79bb-4f24-bb95-93db4ccb4805 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:05.723285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617f48a-79bb-4f24-bb95-93db4ccb4805 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:05.729206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731cd35b-5695-42be-b5b6-eaa73900cd5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=878b71f291ec4107a09e3109b6acc40b, user_id=4d7c1adbb2ad456eaf5793a15d6fcb4b, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:05.731416 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5a257-7f44-41b1-bd1c-586999f73cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:05.732604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731cd35b-5695-42be-b5b6-eaa73900cd5a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:05.733354 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5a257-7f44-41b1-bd1c-586999f73cdb admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:05.733802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5a257-7f44-41b1-bd1c-586999f73cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:05.734079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5a257-7f44-41b1-bd1c-586999f73cdb admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:05.734872 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f5a257-7f44-41b1-bd1c-586999f73cdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:05.737716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f5a257-7f44-41b1-bd1c-586999f73cdb admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:05.752590 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 810/1604] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:05 2022] PUT /identity/v3/projects/878b71f291ec4107a09e3109b6acc40b/users/4d7c1adbb2ad456eaf5793a15d6fcb4b/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:05.754266 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb53c2b-b782-408c-9b27-df1c60605959 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:05.754794 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb53c2b-b782-408c-9b27-df1c60605959 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:05.755136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb53c2b-b782-408c-9b27-df1c60605959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:05.758947 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 795/1605] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:05.763593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06446d-3f2c-4e3c-8af4-94c4202cb243 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:05.764409 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06446d-3f2c-4e3c-8af4-94c4202cb243 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:05.765261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06446d-3f2c-4e3c-8af4-94c4202cb243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:05.786446 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-2bb53c2b-b782-408c-9b27-df1c60605959 None None] MFA Rules not processed for user `6584df0ad24c4fbabc162bb9ffc9e3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:05.786446 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-2bb53c2b-b782-408c-9b27-df1c60605959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:05.815146 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-2d06446d-3f2c-4e3c-8af4-94c4202cb243 None None] MFA Rules not processed for user `60c13dd599844219a310f5eb222ef41f`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:05.816606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2d06446d-3f2c-4e3c-8af4-94c4202cb243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:05.859342 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:05.896724 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 811/1606] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:05.898092 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4e433-c229-4138-b2fb-6720ce3e68fb None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:05.898395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4e433-c229-4138-b2fb-6720ce3e68fb None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:05.898752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4e433-c229-4138-b2fb-6720ce3e68fb None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:05.900011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 812/1607] 10.222.0.36 () {54 vars in 961 bytes} [Wed Jun 22 11:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:45:05.901283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4e433-c229-4138-b2fb-6720ce3e68fb None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:05.926488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 796/1608] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:05.927733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06446d-3f2c-4e3c-8af4-94c4202cb243 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:05.929205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2d06446d-3f2c-4e3c-8af4-94c4202cb243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:05.999470 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:06.008121 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4e433-c229-4138-b2fb-6720ce3e68fb None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:06.014885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474731e-3f6c-4170-9435-00633bb8578d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:06.016308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474731e-3f6c-4170-9435-00633bb8578d admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:06.016308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474731e-3f6c-4170-9435-00633bb8578d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:06.016687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474731e-3f6c-4170-9435-00633bb8578d admin admin] PATH_INFO: `/v3/projects/3e28a026296942abb83826065af4a110/users/de76a5ea7fbb427090f4df4e6424f5a6/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:06.034575 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474731e-3f6c-4170-9435-00633bb8578d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e28a026296942abb83826065af4a110, user_id=de76a5ea7fbb427090f4df4e6424f5a6, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:06.035939 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06446d-3f2c-4e3c-8af4-94c4202cb243 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:06.037715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474731e-3f6c-4170-9435-00633bb8578d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:06.044804 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630406b8-3743-4702-8770-be7fd5b6ab51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:06.045734 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630406b8-3743-4702-8770-be7fd5b6ab51 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:06.046200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630406b8-3743-4702-8770-be7fd5b6ab51 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:06.046608 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630406b8-3743-4702-8770-be7fd5b6ab51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:06.056574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 813/1609] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:05 2022] PUT /identity/v3/projects/3e28a026296942abb83826065af4a110/users/de76a5ea7fbb427090f4df4e6424f5a6/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:06.057957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c56c2f-d085-4ab3-bfbf-e57393f67bdf None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:06.058256 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c56c2f-d085-4ab3-bfbf-e57393f67bdf None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:06.058540 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c56c2f-d085-4ab3-bfbf-e57393f67bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:06.063829 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630406b8-3743-4702-8770-be7fd5b6ab51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:06.065330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630406b8-3743-4702-8770-be7fd5b6ab51 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:06.077092 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 797/1610] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:06.078309 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630406b8-3743-4702-8770-be7fd5b6ab51 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:06.080334 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-c1c56c2f-d085-4ab3-bfbf-e57393f67bdf None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:06.081661 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-c1c56c2f-d085-4ab3-bfbf-e57393f67bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:06.090006 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630406b8-3743-4702-8770-be7fd5b6ab51 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:06.213227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e88b4-a58c-4b10-a748-0c406aa14da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:06.213227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e88b4-a58c-4b10-a748-0c406aa14da3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:06.213227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e88b4-a58c-4b10-a748-0c406aa14da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:06.213227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e88b4-a58c-4b10-a748-0c406aa14da3 admin admin] PATH_INFO: `/v3/projects/4037b91404bd493095b736a4f8db43c5/users/b598b69364134e64a6e1c9e9fc140880/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:06.223848 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e88b4-a58c-4b10-a748-0c406aa14da3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4037b91404bd493095b736a4f8db43c5, user_id=b598b69364134e64a6e1c9e9fc140880, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:06.227972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 814/1611] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:06.228989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c56c2f-d085-4ab3-bfbf-e57393f67bdf None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:06.229592 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e88b4-a58c-4b10-a748-0c406aa14da3 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:06.263273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c56c2f-d085-4ab3-bfbf-e57393f67bdf None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:06.276811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 798/1612] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:06 2022] PUT /identity/v3/projects/4037b91404bd493095b736a4f8db43c5/users/b598b69364134e64a6e1c9e9fc140880/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:06.276811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b8a5c-7975-4035-b111-75052ddf7c1e None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:06.276811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b8a5c-7975-4035-b111-75052ddf7c1e None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:06.276811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b8a5c-7975-4035-b111-75052ddf7c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:06.296658 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-f42b8a5c-7975-4035-b111-75052ddf7c1e None None] MFA Rules not processed for user `479a376dcac14c509c1c55f0046d88cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:06.297952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f42b8a5c-7975-4035-b111-75052ddf7c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:06.377961 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6286f26-da60-47da-98e7-eafc9bfe3fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:06.378655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6286f26-da60-47da-98e7-eafc9bfe3fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:06.378940 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6286f26-da60-47da-98e7-eafc9bfe3fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:06.379216 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6286f26-da60-47da-98e7-eafc9bfe3fc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:06.380141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6286f26-da60-47da-98e7-eafc9bfe3fc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:06.382580 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6286f26-da60-47da-98e7-eafc9bfe3fc8 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:06.409594 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:06.435595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 815/1613] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:06 2022] POST /identity/v3/projects => generated 363 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:06.438152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edfba4-3248-4c54-98f7-d5692d3f5dc0 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:06.438802 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edfba4-3248-4c54-98f7-d5692d3f5dc0 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:06.439189 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edfba4-3248-4c54-98f7-d5692d3f5dc0 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:06.440993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 816/1614] 10.222.0.36 () {54 vars in 961 bytes} [Wed Jun 22 11:45:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:45:06.442282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7091a-1d45-4636-b17f-f40c84c9206b None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:06.442605 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7091a-1d45-4636-b17f-f40c84c9206b None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:06.442954 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7091a-1d45-4636-b17f-f40c84c9206b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:06.449131 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 799/1615] 10.222.0.36 () {58 vars in 1058 bytes} [Wed Jun 22 11:45:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:06.449943 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b8a5c-7975-4035-b111-75052ddf7c1e None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:06.500845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-1af7091a-1d45-4636-b17f-f40c84c9206b None None] MFA Rules not processed for user `de76a5ea7fbb427090f4df4e6424f5a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:06.502185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-1af7091a-1d45-4636-b17f-f40c84c9206b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:06.539440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:06.553995 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b8a5c-7975-4035-b111-75052ddf7c1e None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:06.625045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 817/1616] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:06.626605 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bbfb0-66a9-4acf-b33f-b064f7ca251c None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:06.626898 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bbfb0-66a9-4acf-b33f-b064f7ca251c None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:06.627181 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bbfb0-66a9-4acf-b33f-b064f7ca251c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:06.639691 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10d266-17a8-4f2f-891b-cfece7d8005f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:06.641068 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10d266-17a8-4f2f-891b-cfece7d8005f service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:06.641617 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10d266-17a8-4f2f-891b-cfece7d8005f service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:06.642799 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10d266-17a8-4f2f-891b-cfece7d8005f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:06.663804 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-2c0bbfb0-66a9-4acf-b33f-b064f7ca251c None None] MFA Rules not processed for user `b598b69364134e64a6e1c9e9fc140880`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:06.665799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-2c0bbfb0-66a9-4acf-b33f-b064f7ca251c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:06.722867 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10d266-17a8-4f2f-891b-cfece7d8005f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:06.725145 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d10d266-17a8-4f2f-891b-cfece7d8005f service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:06.739490 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 800/1617] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:06.741763 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10d266-17a8-4f2f-891b-cfece7d8005f service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:06.758418 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10d266-17a8-4f2f-891b-cfece7d8005f service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:06.765058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 818/1618] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:06.767787 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e6eeb-f6b7-4b33-9216-0cea36b3323f None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:06.769342 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e6eeb-f6b7-4b33-9216-0cea36b3323f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:06.770313 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e6eeb-f6b7-4b33-9216-0cea36b3323f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:06.772379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffaca0-4c20-4064-953e-194da740c36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:06.773081 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffaca0-4c20-4064-953e-194da740c36a admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:06.773349 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffaca0-4c20-4064-953e-194da740c36a admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:06.773628 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffaca0-4c20-4064-953e-194da740c36a admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:06.774318 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffaca0-4c20-4064-953e-194da740c36a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:06.782290 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffaca0-4c20-4064-953e-194da740c36a admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:06.796759 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-793e6eeb-f6b7-4b33-9216-0cea36b3323f None None] MFA Rules not processed for user `479a376dcac14c509c1c55f0046d88cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:06.798081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-793e6eeb-f6b7-4b33-9216-0cea36b3323f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:06.827733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 801/1619] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:06 2022] POST /identity/v3/users => generated 288 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:06.828596 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffaca0-4c20-4064-953e-194da740c36a admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:06.947142 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 819/1620] 10.222.0.36 () {58 vars in 1058 bytes} [Wed Jun 22 11:45:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:06.957385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e6eeb-f6b7-4b33-9216-0cea36b3323f None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:06.972928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e6eeb-f6b7-4b33-9216-0cea36b3323f None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:06.984477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9088c-27e3-4213-977f-2633b9fe242f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:06.985525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9088c-27e3-4213-977f-2633b9fe242f admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:06.986030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9088c-27e3-4213-977f-2633b9fe242f admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:06.986522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9088c-27e3-4213-977f-2633b9fe242f admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:06.987497 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9088c-27e3-4213-977f-2633b9fe242f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:06.991778 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9088c-27e3-4213-977f-2633b9fe242f admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:06.994762 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffaca0-4c20-4064-953e-194da740c36a admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:06.996413 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:07.009307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a62f9-2110-4900-9646-b23ac6cac1ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:07.010174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a62f9-2110-4900-9646-b23ac6cac1ee service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:07.010522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a62f9-2110-4900-9646-b23ac6cac1ee service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:07.011011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a62f9-2110-4900-9646-b23ac6cac1ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:07.046215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 820/1621] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:06 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:07.047662 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9088c-27e3-4213-977f-2633b9fe242f admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:07.096578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:07.168686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a62f9-2110-4900-9646-b23ac6cac1ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:07.170406 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a62f9-2110-4900-9646-b23ac6cac1ee service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:07.197081 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 802/1622] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:07.198842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a62f9-2110-4900-9646-b23ac6cac1ee service nova] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:07.223030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9088c-27e3-4213-977f-2633b9fe242f admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:07.228909 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a62f9-2110-4900-9646-b23ac6cac1ee service nova] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:07.237631 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ed6d6-9e7b-464b-9d88-c4a905f6afd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:07.238314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ed6d6-9e7b-464b-9d88-c4a905f6afd7 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:07.238657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ed6d6-9e7b-464b-9d88-c4a905f6afd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:07.238938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ed6d6-9e7b-464b-9d88-c4a905f6afd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:07.239658 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ed6d6-9e7b-464b-9d88-c4a905f6afd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:07.241591 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ed6d6-9e7b-464b-9d88-c4a905f6afd7 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:07.247815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51778b17-4183-4c45-a012-50195ef014c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:07.256500 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51778b17-4183-4c45-a012-50195ef014c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:07.259209 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51778b17-4183-4c45-a012-50195ef014c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:07.259699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51778b17-4183-4c45-a012-50195ef014c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:07.261265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51778b17-4183-4c45-a012-50195ef014c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:07.274456 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51778b17-4183-4c45-a012-50195ef014c9 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:07.303045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 803/1623] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:07.304032 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51778b17-4183-4c45-a012-50195ef014c9 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:07.320410 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 821/1624] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:07 2022] POST /identity/v3/projects => generated 363 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:07.322003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ed6d6-9e7b-464b-9d88-c4a905f6afd7 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:07.458905 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ed6d6-9e7b-464b-9d88-c4a905f6afd7 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:07.473231 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab552edc-f3c9-4600-a9e6-4b1a2feb42aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:07.474446 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab552edc-f3c9-4600-a9e6-4b1a2feb42aa admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:07.475584 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab552edc-f3c9-4600-a9e6-4b1a2feb42aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:07.476628 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab552edc-f3c9-4600-a9e6-4b1a2feb42aa admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:07.478532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51778b17-4183-4c45-a012-50195ef014c9 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:07.479246 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab552edc-f3c9-4600-a9e6-4b1a2feb42aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:07.484755 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab552edc-f3c9-4600-a9e6-4b1a2feb42aa admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:07.543427 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 822/1625] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:07 2022] POST /identity/v3/users => generated 292 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:07.550681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab552edc-f3c9-4600-a9e6-4b1a2feb42aa admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:07.567955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab552edc-f3c9-4600-a9e6-4b1a2feb42aa admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:07.575111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da50ea-f0b9-4881-ac5b-806a2a518f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:07.576678 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da50ea-f0b9-4881-ac5b-806a2a518f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:07.577214 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da50ea-f0b9-4881-ac5b-806a2a518f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:07.577706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da50ea-f0b9-4881-ac5b-806a2a518f33 admin admin] PATH_INFO: `/v3/projects/a4364d831dd84245ac42f4f461f74c74/users/0c66b16388734ae98ad2b16104da0bb7/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:07.622958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da50ea-f0b9-4881-ac5b-806a2a518f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4364d831dd84245ac42f4f461f74c74, user_id=0c66b16388734ae98ad2b16104da0bb7, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:07.627879 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625da882-5437-47b5-8e68-95ca5dbdf706 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:07.628565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625da882-5437-47b5-8e68-95ca5dbdf706 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:07.628849 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625da882-5437-47b5-8e68-95ca5dbdf706 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:07.629127 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625da882-5437-47b5-8e68-95ca5dbdf706 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:07.630995 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da50ea-f0b9-4881-ac5b-806a2a518f33 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:07.679393 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 823/1626] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:07 2022] PUT /identity/v3/projects/a4364d831dd84245ac42f4f461f74c74/users/0c66b16388734ae98ad2b16104da0bb7/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:07.681444 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da50ea-f0b9-4881-ac5b-806a2a518f33 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:07.744777 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:07.752157 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625da882-5437-47b5-8e68-95ca5dbdf706 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:07.757430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625da882-5437-47b5-8e68-95ca5dbdf706 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:07.760638 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:07.789940 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da50ea-f0b9-4881-ac5b-806a2a518f33 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:07.792377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 804/1627] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:07.796977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a7807-fa8c-4273-b870-1955014ad3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:07.799409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a7807-fa8c-4273-b870-1955014ad3cd admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:07.799409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a7807-fa8c-4273-b870-1955014ad3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:07.799409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a7807-fa8c-4273-b870-1955014ad3cd admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:07.799968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a7807-fa8c-4273-b870-1955014ad3cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:07.802034 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a7807-fa8c-4273-b870-1955014ad3cd admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:07.803095 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625da882-5437-47b5-8e68-95ca5dbdf706 service nova] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:07.841622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625da882-5437-47b5-8e68-95ca5dbdf706 service nova] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:07.853936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 824/1628] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:07 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:07.853936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a7807-fa8c-4273-b870-1955014ad3cd admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:07.863745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc3b47-4259-4278-9bd7-621c4b815fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:07.864437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc3b47-4259-4278-9bd7-621c4b815fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:07.864717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc3b47-4259-4278-9bd7-621c4b815fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:07.865030 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc3b47-4259-4278-9bd7-621c4b815fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:07.865767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc3b47-4259-4278-9bd7-621c4b815fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:07.871074 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc3b47-4259-4278-9bd7-621c4b815fa8 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:07.873827 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a7807-fa8c-4273-b870-1955014ad3cd admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:07.894229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe20401-2f37-492b-9bd7-68920d9829ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:07.895118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe20401-2f37-492b-9bd7-68920d9829ce admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:07.895580 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe20401-2f37-492b-9bd7-68920d9829ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:07.895961 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe20401-2f37-492b-9bd7-68920d9829ce admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:07.896841 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe20401-2f37-492b-9bd7-68920d9829ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:07.898552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe20401-2f37-492b-9bd7-68920d9829ce admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:07.912384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 805/1629] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:07.912384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc3b47-4259-4278-9bd7-621c4b815fa8 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:07.916457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 825/1630] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:07.930304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60b638-39c4-4651-819e-affa647b8de4 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:07.930304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60b638-39c4-4651-819e-affa647b8de4 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:07.930304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60b638-39c4-4651-819e-affa647b8de4 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:07.935063 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 826/1631] 10.222.0.36 () {54 vars in 912 bytes} [Wed Jun 22 11:45:07 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:45:07.935630 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc3b47-4259-4278-9bd7-621c4b815fa8 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:07.936747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60b638-39c4-4651-819e-affa647b8de4 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:07.951398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51854e-f0ff-40c7-ba01-78d902faee74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:07.952220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51854e-f0ff-40c7-ba01-78d902faee74 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:07.952509 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51854e-f0ff-40c7-ba01-78d902faee74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:07.952787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51854e-f0ff-40c7-ba01-78d902faee74 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:07.953529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51854e-f0ff-40c7-ba01-78d902faee74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:07.956312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51854e-f0ff-40c7-ba01-78d902faee74 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:07.978824 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 806/1632] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:07.983945 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60b638-39c4-4651-819e-affa647b8de4 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:07.985631 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51854e-f0ff-40c7-ba01-78d902faee74 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:08.001076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98c2f5-879d-47a4-9d6f-f4a07be9e2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:08.002168 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98c2f5-879d-47a4-9d6f-f4a07be9e2b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:08.002555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98c2f5-879d-47a4-9d6f-f4a07be9e2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:08.002951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98c2f5-879d-47a4-9d6f-f4a07be9e2b9 admin admin] PATH_INFO: `/v3/projects/4689468bb3224ac68a6e3e6b4067b947/users/2781d98232fc4a90bce29a7d5ba5b9ed/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:08.004067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51854e-f0ff-40c7-ba01-78d902faee74 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:08.021065 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b37b2-b793-4475-942e-cb934f1af875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:08.022022 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b37b2-b793-4475-942e-cb934f1af875 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:08.022362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b37b2-b793-4475-942e-cb934f1af875 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:08.022616 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b37b2-b793-4475-942e-cb934f1af875 admin admin] PATH_INFO: `/v3/projects/a4364d831dd84245ac42f4f461f74c74/users/0c66b16388734ae98ad2b16104da0bb7/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:08.031415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b37b2-b793-4475-942e-cb934f1af875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4364d831dd84245ac42f4f461f74c74, user_id=0c66b16388734ae98ad2b16104da0bb7, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:08.034706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b37b2-b793-4475-942e-cb934f1af875 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:08.056821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98c2f5-879d-47a4-9d6f-f4a07be9e2b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4689468bb3224ac68a6e3e6b4067b947, user_id=2781d98232fc4a90bce29a7d5ba5b9ed, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:08.069112 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 807/1633] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:07 2022] PUT /identity/v3/projects/a4364d831dd84245ac42f4f461f74c74/users/0c66b16388734ae98ad2b16104da0bb7/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:08.069112 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2b6ba-f5e7-4e4c-a1cf-14aa2964a278 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:08.069112 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2b6ba-f5e7-4e4c-a1cf-14aa2964a278 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:08.069997 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98c2f5-879d-47a4-9d6f-f4a07be9e2b9 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:08.070151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2b6ba-f5e7-4e4c-a1cf-14aa2964a278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:08.102684 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-a1c2b6ba-f5e7-4e4c-a1cf-14aa2964a278 None None] MFA Rules not processed for user `479a376dcac14c509c1c55f0046d88cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:08.104406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 827/1634] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:07 2022] PUT /identity/v3/projects/4689468bb3224ac68a6e3e6b4067b947/users/2781d98232fc4a90bce29a7d5ba5b9ed/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:08.104621 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a1c2b6ba-f5e7-4e4c-a1cf-14aa2964a278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:08.105125 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98c2f5-879d-47a4-9d6f-f4a07be9e2b9 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:08.258990 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98c2f5-879d-47a4-9d6f-f4a07be9e2b9 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:08.264932 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efabd66-ee7c-4c70-a737-1a59afad4f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:08.265440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 808/1635] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jun 22 11:45:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:08.266651 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efabd66-ee7c-4c70-a737-1a59afad4f31 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:08.267495 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efabd66-ee7c-4c70-a737-1a59afad4f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:08.267626 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e392f-13c6-445e-9481-67a424c09b85 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:08.267796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efabd66-ee7c-4c70-a737-1a59afad4f31 admin admin] PATH_INFO: `/v3/projects/c1d90ace493343819f5874743800f570/users/1d360d4ce5dd4237bdb7154ec5dba559/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:08.267903 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e392f-13c6-445e-9481-67a424c09b85 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:08.268079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e392f-13c6-445e-9481-67a424c09b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:08.291960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efabd66-ee7c-4c70-a737-1a59afad4f31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1d90ace493343819f5874743800f570, user_id=1d360d4ce5dd4237bdb7154ec5dba559, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:08.298602 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efabd66-ee7c-4c70-a737-1a59afad4f31 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:08.324946 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-345e392f-13c6-445e-9481-67a424c09b85 None None] MFA Rules not processed for user `0c66b16388734ae98ad2b16104da0bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:08.326152 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-345e392f-13c6-445e-9481-67a424c09b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:08.328176 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 828/1636] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:08 2022] PUT /identity/v3/projects/c1d90ace493343819f5874743800f570/users/1d360d4ce5dd4237bdb7154ec5dba559/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:08.333024 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efabd66-ee7c-4c70-a737-1a59afad4f31 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:08.469430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 809/1637] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:08 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:08.470613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e392f-13c6-445e-9481-67a424c09b85 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:08.511098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efabd66-ee7c-4c70-a737-1a59afad4f31 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:08.525929 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b41c8f-d654-4574-bb9e-0f1ce06b4662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:08.527853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:08.529692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b41c8f-d654-4574-bb9e-0f1ce06b4662 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:08.529979 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b41c8f-d654-4574-bb9e-0f1ce06b4662 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:08.530262 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b41c8f-d654-4574-bb9e-0f1ce06b4662 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:08.531020 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b41c8f-d654-4574-bb9e-0f1ce06b4662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:08.532768 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b41c8f-d654-4574-bb9e-0f1ce06b4662 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:08.548820 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e392f-13c6-445e-9481-67a424c09b85 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:08.553675 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31434ea-2602-48a1-bb53-4eced8d583fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:08.554494 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31434ea-2602-48a1-bb53-4eced8d583fd service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:08.554875 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31434ea-2602-48a1-bb53-4eced8d583fd service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:08.555425 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31434ea-2602-48a1-bb53-4eced8d583fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:08.568148 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 829/1638] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:08.569025 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b41c8f-d654-4574-bb9e-0f1ce06b4662 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:08.589968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b41c8f-d654-4574-bb9e-0f1ce06b4662 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:08.601287 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57af3e-6da3-43a8-bb83-5d2748f52579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:08.601977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57af3e-6da3-43a8-bb83-5d2748f52579 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:08.602260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57af3e-6da3-43a8-bb83-5d2748f52579 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:08.602539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57af3e-6da3-43a8-bb83-5d2748f52579 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:08.603272 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac57af3e-6da3-43a8-bb83-5d2748f52579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:08.606527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac57af3e-6da3-43a8-bb83-5d2748f52579 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:08.612838 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:08.639397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 830/1639] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:08.639397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f0772-92bb-45a9-80d1-a791b7ffb603 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:08.639397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f0772-92bb-45a9-80d1-a791b7ffb603 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:08.640704 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f0772-92bb-45a9-80d1-a791b7ffb603 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:08.644020 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 831/1640] 10.222.0.36 () {54 vars in 912 bytes} [Wed Jun 22 11:45:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:45:08.647069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f0772-92bb-45a9-80d1-a791b7ffb603 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:08.655026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31434ea-2602-48a1-bb53-4eced8d583fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:08.656777 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31434ea-2602-48a1-bb53-4eced8d583fd service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:08.679913 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f0772-92bb-45a9-80d1-a791b7ffb603 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:08.686958 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 810/1641] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:08.688094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a102f-e2bd-4623-b397-a1ba7c342ded service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:08.689580 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a102f-e2bd-4623-b397-a1ba7c342ded service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:08.689580 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a102f-e2bd-4623-b397-a1ba7c342ded service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:08.691462 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31434ea-2602-48a1-bb53-4eced8d583fd service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:08.692145 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a102f-e2bd-4623-b397-a1ba7c342ded service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:08.713346 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a102f-e2bd-4623-b397-a1ba7c342ded service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:08.717337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31434ea-2602-48a1-bb53-4eced8d583fd service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:08.718633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a102f-e2bd-4623-b397-a1ba7c342ded service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:08.727979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5c45e-3ece-4b0c-a036-8b9c11abecee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:08.733116 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5c45e-3ece-4b0c-a036-8b9c11abecee admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:08.733697 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5c45e-3ece-4b0c-a036-8b9c11abecee admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:08.734184 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5c45e-3ece-4b0c-a036-8b9c11abecee admin admin] PATH_INFO: `/v3/projects/4689468bb3224ac68a6e3e6b4067b947/users/2781d98232fc4a90bce29a7d5ba5b9ed/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:08.739644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5c45e-3ece-4b0c-a036-8b9c11abecee admin admin] RBAC: Authorizing `identity:create_grant(project_id=4689468bb3224ac68a6e3e6b4067b947, user_id=2781d98232fc4a90bce29a7d5ba5b9ed, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:08.743382 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 832/1642] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:08 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:08.744899 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a102f-e2bd-4623-b397-a1ba7c342ded service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:08.759804 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5c45e-3ece-4b0c-a036-8b9c11abecee admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:08.779703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a102f-e2bd-4623-b397-a1ba7c342ded service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:08.788120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d07ee50-e2b3-4ca8-9265-b68fa8d028b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:08.789303 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07ee50-e2b3-4ca8-9265-b68fa8d028b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:08.789303 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07ee50-e2b3-4ca8-9265-b68fa8d028b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:08.789583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d07ee50-e2b3-4ca8-9265-b68fa8d028b9 admin admin] PATH_INFO: `/v3/projects/c1d90ace493343819f5874743800f570/users/1d360d4ce5dd4237bdb7154ec5dba559/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:08.792193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07ee50-e2b3-4ca8-9265-b68fa8d028b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1d90ace493343819f5874743800f570, user_id=1d360d4ce5dd4237bdb7154ec5dba559, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:08.797571 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d07ee50-e2b3-4ca8-9265-b68fa8d028b9 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:08.807320 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 811/1643] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:08 2022] PUT /identity/v3/projects/4689468bb3224ac68a6e3e6b4067b947/users/2781d98232fc4a90bce29a7d5ba5b9ed/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:08.811502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1cafa-4f12-4e96-909c-3aec54c69812 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:08.811790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1cafa-4f12-4e96-909c-3aec54c69812 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:08.814944 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1cafa-4f12-4e96-909c-3aec54c69812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:08.841578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 833/1644] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:08 2022] PUT /identity/v3/projects/c1d90ace493343819f5874743800f570/users/1d360d4ce5dd4237bdb7154ec5dba559/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:08.842855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bb8c6-9951-4869-bd6b-52f94209d1f7 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:08.843145 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bb8c6-9951-4869-bd6b-52f94209d1f7 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:08.848085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bb8c6-9951-4869-bd6b-52f94209d1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:08.887320 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-79d1cafa-4f12-4e96-909c-3aec54c69812 None None] MFA Rules not processed for user `479a376dcac14c509c1c55f0046d88cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:08.888610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-79d1cafa-4f12-4e96-909c-3aec54c69812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:08.906443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-5a3bb8c6-9951-4869-bd6b-52f94209d1f7 None None] MFA Rules not processed for user `2781d98232fc4a90bce29a7d5ba5b9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:08.907966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5a3bb8c6-9951-4869-bd6b-52f94209d1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:09.113992 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 834/1645] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:08 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:09.115803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f878bb5-de14-4972-9778-8f644e32972a None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:09.116094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f878bb5-de14-4972-9778-8f644e32972a None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:09.116419 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f878bb5-de14-4972-9778-8f644e32972a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:09.154335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 812/1646] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jun 22 11:45:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:09.155336 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d1cafa-4f12-4e96-909c-3aec54c69812 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:09.200068 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-2f878bb5-de14-4972-9778-8f644e32972a None None] MFA Rules not processed for user `1d360d4ce5dd4237bdb7154ec5dba559`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:09.204230 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-2f878bb5-de14-4972-9778-8f644e32972a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:09.265506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d1cafa-4f12-4e96-909c-3aec54c69812 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:09.275319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e7c66-c02e-4ce3-9d9f-37c9f7e407e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:09.276096 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e7c66-c02e-4ce3-9d9f-37c9f7e407e1 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:09.276218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e7c66-c02e-4ce3-9d9f-37c9f7e407e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:09.276479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e7c66-c02e-4ce3-9d9f-37c9f7e407e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:09.296479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e7c66-c02e-4ce3-9d9f-37c9f7e407e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:09.297939 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e7c66-c02e-4ce3-9d9f-37c9f7e407e1 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:09.302949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:09.323625 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 813/1647] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:09 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:09.404254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e7c66-c02e-4ce3-9d9f-37c9f7e407e1 service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:09.417161 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e7c66-c02e-4ce3-9d9f-37c9f7e407e1 service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:09.419765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 835/1648] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:09 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:09.422739 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79859c3a-fa72-4066-935d-7a29fc1a2109 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:09.424293 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79859c3a-fa72-4066-935d-7a29fc1a2109 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:09.424766 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79859c3a-fa72-4066-935d-7a29fc1a2109 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:09.425450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79859c3a-fa72-4066-935d-7a29fc1a2109 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:09.444233 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79859c3a-fa72-4066-935d-7a29fc1a2109 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:09.446439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79859c3a-fa72-4066-935d-7a29fc1a2109 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:09.487706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 814/1649] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:09.735748 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f878bb5-de14-4972-9778-8f644e32972a None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:09.749561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f878bb5-de14-4972-9778-8f644e32972a None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:09.776861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43a5ba-015d-4498-a6b6-c94dee159139 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:09.781229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43a5ba-015d-4498-a6b6-c94dee159139 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:09.781631 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43a5ba-015d-4498-a6b6-c94dee159139 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:09.781922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43a5ba-015d-4498-a6b6-c94dee159139 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:09.802806 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf43a5ba-015d-4498-a6b6-c94dee159139 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:09.808743 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf43a5ba-015d-4498-a6b6-c94dee159139 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:09.854506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 836/1650] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:09 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:10.209649 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79859c3a-fa72-4066-935d-7a29fc1a2109 service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:10.210168 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43a5ba-015d-4498-a6b6-c94dee159139 service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:10.223308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43a5ba-015d-4498-a6b6-c94dee159139 service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:10.244836 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104e18b-c7a9-4259-9c46-6a00bf2c0e02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:10.245698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104e18b-c7a9-4259-9c46-6a00bf2c0e02 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:10.245995 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104e18b-c7a9-4259-9c46-6a00bf2c0e02 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:10.246281 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104e18b-c7a9-4259-9c46-6a00bf2c0e02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:10.365269 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:10.425404 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79859c3a-fa72-4066-935d-7a29fc1a2109 service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:10.437815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e4c86-bc0a-473e-907f-76da923936e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:10.442298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e4c86-bc0a-473e-907f-76da923936e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:10.443132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e4c86-bc0a-473e-907f-76da923936e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:10.444057 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e4c86-bc0a-473e-907f-76da923936e1 admin admin] PATH_INFO: `/v3/users/60c13dd599844219a310f5eb222ef41f` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:10.447366 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e4c86-bc0a-473e-907f-76da923936e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60c13dd599844219a310f5eb222ef41f)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:10.452726 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e4c86-bc0a-473e-907f-76da923936e1 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:10.473569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5104e18b-c7a9-4259-9c46-6a00bf2c0e02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:10.478163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5104e18b-c7a9-4259-9c46-6a00bf2c0e02 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:10.514405 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 837/1651] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:10 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:45:10.571843 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ea8e4c86-bc0a-473e-907f-76da923936e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60c13dd599844219a310f5eb222ef41f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:10.605410 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ea8e4c86-bc0a-473e-907f-76da923936e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60c13dd599844219a310f5eb222ef41f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:10.616481 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ea8e4c86-bc0a-473e-907f-76da923936e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60c13dd599844219a310f5eb222ef41f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:10.618025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ea8e4c86-bc0a-473e-907f-76da923936e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60c13dd599844219a310f5eb222ef41f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:10.636236 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ea8e4c86-bc0a-473e-907f-76da923936e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60c13dd599844219a310f5eb222ef41f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:10.649611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 815/1652] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:10 2022] DELETE /identity/v3/users/60c13dd599844219a310f5eb222ef41f => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:10.654469 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba80ff-ca74-4120-accb-1dc5c7636800 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:10.654768 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba80ff-ca74-4120-accb-1dc5c7636800 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:10.655052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba80ff-ca74-4120-accb-1dc5c7636800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:10.687709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-feba80ff-ca74-4120-accb-1dc5c7636800 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:10.689562 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-feba80ff-ca74-4120-accb-1dc5c7636800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:10.904151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 838/1653] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:11.147491 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e4c86-bc0a-473e-907f-76da923936e1 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:11.149657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ea8e4c86-bc0a-473e-907f-76da923936e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:11.233058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba80ff-ca74-4120-accb-1dc5c7636800 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:11.239198 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-feba80ff-ca74-4120-accb-1dc5c7636800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:11.265581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e4c86-bc0a-473e-907f-76da923936e1 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:11.268751 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba80ff-ca74-4120-accb-1dc5c7636800 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:11.269815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415bcc4-e9f9-4c9c-bafc-a6ea863fb676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:11.270475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415bcc4-e9f9-4c9c-bafc-a6ea863fb676 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:11.270761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415bcc4-e9f9-4c9c-bafc-a6ea863fb676 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:11.271038 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415bcc4-e9f9-4c9c-bafc-a6ea863fb676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:11.275390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039d2cb-0a7a-4f5e-9b14-a293404b5d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:11.276600 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039d2cb-0a7a-4f5e-9b14-a293404b5d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:11.277049 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039d2cb-0a7a-4f5e-9b14-a293404b5d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:11.277490 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039d2cb-0a7a-4f5e-9b14-a293404b5d25 admin admin] PATH_INFO: `/v3/users/6584df0ad24c4fbabc162bb9ffc9e3f4` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:11.278976 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039d2cb-0a7a-4f5e-9b14-a293404b5d25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6584df0ad24c4fbabc162bb9ffc9e3f4)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:11.280959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: WARNING py.warnings [None req-6039d2cb-0a7a-4f5e-9b14-a293404b5d25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 22 11:45:11.280959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: warnings.warn(msg) Jun 22 11:45:11.280959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: Jun 22 11:45:11.281981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415bcc4-e9f9-4c9c-bafc-a6ea863fb676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:11.283732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039d2cb-0a7a-4f5e-9b14-a293404b5d25 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:11.284392 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415bcc4-e9f9-4c9c-bafc-a6ea863fb676 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:11.306509 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 816/1654] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:11.322500 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415bcc4-e9f9-4c9c-bafc-a6ea863fb676 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:11.324135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-1415bcc4-e9f9-4c9c-bafc-a6ea863fb676 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:11.346761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415bcc4-e9f9-4c9c-bafc-a6ea863fb676 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:11.362367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d33a5-fbd9-4ed7-9f93-455c001d5555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:11.363399 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d33a5-fbd9-4ed7-9f93-455c001d5555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:11.363778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d33a5-fbd9-4ed7-9f93-455c001d5555 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:11.364124 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d33a5-fbd9-4ed7-9f93-455c001d5555 admin admin] PATH_INFO: `/v3/users/de76a5ea7fbb427090f4df4e6424f5a6` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:11.365695 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d33a5-fbd9-4ed7-9f93-455c001d5555 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de76a5ea7fbb427090f4df4e6424f5a6)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:11.367926 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d33a5-fbd9-4ed7-9f93-455c001d5555 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:11.371536 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:11.430998 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6039d2cb-0a7a-4f5e-9b14-a293404b5d25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6584df0ad24c4fbabc162bb9ffc9e3f4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:11.459536 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ac4d33a5-fbd9-4ed7-9f93-455c001d5555 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de76a5ea7fbb427090f4df4e6424f5a6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:11.483083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ac4d33a5-fbd9-4ed7-9f93-455c001d5555 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de76a5ea7fbb427090f4df4e6424f5a6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:11.488829 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6039d2cb-0a7a-4f5e-9b14-a293404b5d25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6584df0ad24c4fbabc162bb9ffc9e3f4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:11.494199 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ac4d33a5-fbd9-4ed7-9f93-455c001d5555 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de76a5ea7fbb427090f4df4e6424f5a6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:11.496766 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ac4d33a5-fbd9-4ed7-9f93-455c001d5555 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de76a5ea7fbb427090f4df4e6424f5a6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:11.510078 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6039d2cb-0a7a-4f5e-9b14-a293404b5d25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6584df0ad24c4fbabc162bb9ffc9e3f4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:11.523924 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ac4d33a5-fbd9-4ed7-9f93-455c001d5555 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de76a5ea7fbb427090f4df4e6424f5a6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:11.537254 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6039d2cb-0a7a-4f5e-9b14-a293404b5d25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6584df0ad24c4fbabc162bb9ffc9e3f4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:11.542080 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 817/1655] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:11 2022] DELETE /identity/v3/users/de76a5ea7fbb427090f4df4e6424f5a6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:45:11.543445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6039d2cb-0a7a-4f5e-9b14-a293404b5d25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6584df0ad24c4fbabc162bb9ffc9e3f4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:11.544232 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3372a35-d2b0-4a17-aa21-c81c00596ead None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:11.544527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3372a35-d2b0-4a17-aa21-c81c00596ead None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:11.544808 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3372a35-d2b0-4a17-aa21-c81c00596ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:11.567657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 839/1656] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:11 2022] DELETE /identity/v3/users/6584df0ad24c4fbabc162bb9ffc9e3f4 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:11.574754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c4deb-a4f4-40f5-bcd0-da31ba905ce2 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:11.575240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c4deb-a4f4-40f5-bcd0-da31ba905ce2 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:11.575847 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c4deb-a4f4-40f5-bcd0-da31ba905ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:11.603259 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-a3372a35-d2b0-4a17-aa21-c81c00596ead None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:11.606868 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a3372a35-d2b0-4a17-aa21-c81c00596ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:11.620548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-144c4deb-a4f4-40f5-bcd0-da31ba905ce2 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:11.623704 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-144c4deb-a4f4-40f5-bcd0-da31ba905ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:11.762900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:11.817617 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 818/1657] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:11.866572 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 840/1658] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:11.918529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3372a35-d2b0-4a17-aa21-c81c00596ead None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:11.931997 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a3372a35-d2b0-4a17-aa21-c81c00596ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:11.932225 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c4deb-a4f4-40f5-bcd0-da31ba905ce2 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:11.935282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-144c4deb-a4f4-40f5-bcd0-da31ba905ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:12.069061 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c4deb-a4f4-40f5-bcd0-da31ba905ce2 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:12.085231 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0c643-0881-44b7-b279-3d8cf35031fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:12.085909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0c643-0881-44b7-b279-3d8cf35031fd service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:12.086195 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0c643-0881-44b7-b279-3d8cf35031fd service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:12.086473 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0c643-0881-44b7-b279-3d8cf35031fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:12.113675 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0c643-0881-44b7-b279-3d8cf35031fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:12.115142 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0c643-0881-44b7-b279-3d8cf35031fd service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:12.146270 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 841/1659] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:12.250387 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3372a35-d2b0-4a17-aa21-c81c00596ead None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:12.265450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b570b-0dcf-42ee-a3a1-6f0222942189 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:12.266558 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b570b-0dcf-42ee-a3a1-6f0222942189 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:12.268412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b570b-0dcf-42ee-a3a1-6f0222942189 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:12.274299 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b570b-0dcf-42ee-a3a1-6f0222942189 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:12.330848 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b570b-0dcf-42ee-a3a1-6f0222942189 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:12.353105 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b570b-0dcf-42ee-a3a1-6f0222942189 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:12.376207 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 819/1660] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:13.305264 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0c643-0881-44b7-b279-3d8cf35031fd service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:13.308148 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-9fc0c643-0881-44b7-b279-3d8cf35031fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:13.342757 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0c643-0881-44b7-b279-3d8cf35031fd service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:13.355596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94758480-cee8-488c-a81c-c9986463c456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:13.357543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94758480-cee8-488c-a81c-c9986463c456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:13.358644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94758480-cee8-488c-a81c-c9986463c456 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:13.361176 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94758480-cee8-488c-a81c-c9986463c456 admin admin] PATH_INFO: `/v3/projects/57c834b746fc4a11aa0226019923d4d0` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:13.366379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94758480-cee8-488c-a81c-c9986463c456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57c834b746fc4a11aa0226019923d4d0)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:13.375151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94758480-cee8-488c-a81c-c9986463c456 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:13.490799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-94758480-cee8-488c-a81c-c9986463c456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57c834b746fc4a11aa0226019923d4d0', 'request_id': 'req-94758480-cee8-488c-a81c-c9986463c456'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:13.503490 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-94758480-cee8-488c-a81c-c9986463c456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57c834b746fc4a11aa0226019923d4d0', 'request_id': 'req-94758480-cee8-488c-a81c-c9986463c456'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:13.516633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-94758480-cee8-488c-a81c-c9986463c456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57c834b746fc4a11aa0226019923d4d0', 'request_id': 'req-94758480-cee8-488c-a81c-c9986463c456'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:13.533518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-94758480-cee8-488c-a81c-c9986463c456 admin admin] The token cache is being invalidate because project 57c834b746fc4a11aa0226019923d4d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:13.534001 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-94758480-cee8-488c-a81c-c9986463c456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:13.534794 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-94758480-cee8-488c-a81c-c9986463c456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:13.536901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 842/1661] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:13 2022] DELETE /identity/v3/projects/57c834b746fc4a11aa0226019923d4d0 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:13.541758 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f63c8b-f81c-4288-b568-c3a5d7642f21 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:13.542942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f63c8b-f81c-4288-b568-c3a5d7642f21 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:13.543533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f63c8b-f81c-4288-b568-c3a5d7642f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:13.584788 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-e9f63c8b-f81c-4288-b568-c3a5d7642f21 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:13.586337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e9f63c8b-f81c-4288-b568-c3a5d7642f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:13.639883 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:13.718993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94758480-cee8-488c-a81c-c9986463c456 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:13.727462 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-94758480-cee8-488c-a81c-c9986463c456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:13.774605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 820/1662] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:13.783703 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbda54-e6e6-4179-b613-40418cd887e6 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:13.784018 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbda54-e6e6-4179-b613-40418cd887e6 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:13.784272 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbda54-e6e6-4179-b613-40418cd887e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:13.850192 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-32cbda54-e6e6-4179-b613-40418cd887e6 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:13.850192 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-32cbda54-e6e6-4179-b613-40418cd887e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:13.878259 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 821/1663] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:13.884801 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbda54-e6e6-4179-b613-40418cd887e6 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:13.887466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94758480-cee8-488c-a81c-c9986463c456 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:13.910082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c318e-470a-4480-bddb-bf71065c2984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:13.911174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbda54-e6e6-4179-b613-40418cd887e6 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:13.911903 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c318e-470a-4480-bddb-bf71065c2984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:13.912703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c318e-470a-4480-bddb-bf71065c2984 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:13.913188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c318e-470a-4480-bddb-bf71065c2984 admin admin] PATH_INFO: `/v3/users/54b2ffa75d014f4691421782afb8664f` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:13.914793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c318e-470a-4480-bddb-bf71065c2984 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54b2ffa75d014f4691421782afb8664f)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:13.917454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c318e-470a-4480-bddb-bf71065c2984 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:13.926461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:13.929451 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53e511-0e24-424e-8151-fbc8af42adf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:13.930147 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53e511-0e24-424e-8151-fbc8af42adf2 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:13.930432 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53e511-0e24-424e-8151-fbc8af42adf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:13.930710 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53e511-0e24-424e-8151-fbc8af42adf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:13.931506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53e511-0e24-424e-8151-fbc8af42adf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:13.931953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53e511-0e24-424e-8151-fbc8af42adf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:13.937561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: WARNING py.warnings [None req-9b53e511-0e24-424e-8151-fbc8af42adf2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 22 11:45:13.937561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: warnings.warn(msg) Jun 22 11:45:13.937561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: Jun 22 11:45:13.938536 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53e511-0e24-424e-8151-fbc8af42adf2 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:13.982337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 822/1664] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:13.986161 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53e511-0e24-424e-8151-fbc8af42adf2 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:14.002279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53e511-0e24-424e-8151-fbc8af42adf2 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:14.019551 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e622e-9260-4300-a8cc-e0b8a231cc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:14.019910 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e622e-9260-4300-a8cc-e0b8a231cc45 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:14.020147 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e622e-9260-4300-a8cc-e0b8a231cc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:14.020431 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e622e-9260-4300-a8cc-e0b8a231cc45 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:14.021146 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e622e-9260-4300-a8cc-e0b8a231cc45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:14.023175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e622e-9260-4300-a8cc-e0b8a231cc45 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:14.039736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-5e6c318e-470a-4480-bddb-bf71065c2984 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54b2ffa75d014f4691421782afb8664f'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:14.076775 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-5e6c318e-470a-4480-bddb-bf71065c2984 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54b2ffa75d014f4691421782afb8664f'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:14.088042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-5e6c318e-470a-4480-bddb-bf71065c2984 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54b2ffa75d014f4691421782afb8664f'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:14.100859 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 823/1665] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:13 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:45:14.102366 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22fc1e-849c-4d67-a5af-ec9bd34bf0c6 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:14.102658 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22fc1e-849c-4d67-a5af-ec9bd34bf0c6 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:14.102941 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22fc1e-849c-4d67-a5af-ec9bd34bf0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:14.104536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-5e6c318e-470a-4480-bddb-bf71065c2984 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54b2ffa75d014f4691421782afb8664f'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:14.105202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-5e6c318e-470a-4480-bddb-bf71065c2984 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54b2ffa75d014f4691421782afb8664f'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:14.114937 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 843/1666] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:13 2022] DELETE /identity/v3/users/54b2ffa75d014f4691421782afb8664f => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:45:14.115891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c318e-470a-4480-bddb-bf71065c2984 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:14.117574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5e6c318e-470a-4480-bddb-bf71065c2984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:14.166196 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-3b22fc1e-849c-4d67-a5af-ec9bd34bf0c6 None None] MFA Rules not processed for user `3b8a3e35ce254868bf3825cf60139979`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:14.169875 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-3b22fc1e-849c-4d67-a5af-ec9bd34bf0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:14.295170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c318e-470a-4480-bddb-bf71065c2984 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:14.309302 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed43bd-02fb-40d0-9fc5-60b4c85db8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:14.310034 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed43bd-02fb-40d0-9fc5-60b4c85db8af admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:14.310274 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed43bd-02fb-40d0-9fc5-60b4c85db8af admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:14.310548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed43bd-02fb-40d0-9fc5-60b4c85db8af admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:14.311249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed43bd-02fb-40d0-9fc5-60b4c85db8af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:14.313985 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed43bd-02fb-40d0-9fc5-60b4c85db8af admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:14.351517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 824/1667] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:14.351517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c70939-656e-4228-abfe-7af57b5680f6 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:14.351517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c70939-656e-4228-abfe-7af57b5680f6 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:14.351517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c70939-656e-4228-abfe-7af57b5680f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:14.370843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 844/1668] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:14 2022] POST /identity/v3/users => generated 287 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:14.373968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396754f2-3112-433b-b13f-c376e499e0f5 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:14.374484 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396754f2-3112-433b-b13f-c376e499e0f5 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:14.375022 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396754f2-3112-433b-b13f-c376e499e0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:14.409459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-91c70939-656e-4228-abfe-7af57b5680f6 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:14.411335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-396754f2-3112-433b-b13f-c376e499e0f5 None None] MFA Rules not processed for user `3b8a3e35ce254868bf3825cf60139979`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:14.414857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-91c70939-656e-4228-abfe-7af57b5680f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:14.415465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-396754f2-3112-433b-b13f-c376e499e0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:14.455079 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 845/1669] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:14.456459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 825/1670] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:14.460739 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396754f2-3112-433b-b13f-c376e499e0f5 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:14.461302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c70939-656e-4228-abfe-7af57b5680f6 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:14.473707 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-91c70939-656e-4228-abfe-7af57b5680f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:14.478156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396754f2-3112-433b-b13f-c376e499e0f5 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:14.489668 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06258458-ae07-4c3c-bb86-547d3be1be21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:14.490516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06258458-ae07-4c3c-bb86-547d3be1be21 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:14.490776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06258458-ae07-4c3c-bb86-547d3be1be21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:14.491060 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06258458-ae07-4c3c-bb86-547d3be1be21 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:14.491856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06258458-ae07-4c3c-bb86-547d3be1be21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:14.494783 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06258458-ae07-4c3c-bb86-547d3be1be21 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:14.520289 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 846/1671] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:14.521213 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06258458-ae07-4c3c-bb86-547d3be1be21 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:14.526398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-06258458-ae07-4c3c-bb86-547d3be1be21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:14.697648 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:14.700234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06258458-ae07-4c3c-bb86-547d3be1be21 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:14.711583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c70939-656e-4228-abfe-7af57b5680f6 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:14.715460 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c5392-854b-4488-9992-fbee5cfe6b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:14.716152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c5392-854b-4488-9992-fbee5cfe6b0d service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:14.716440 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c5392-854b-4488-9992-fbee5cfe6b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:14.716724 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c5392-854b-4488-9992-fbee5cfe6b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:14.725755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c801ea2-ffd3-4e2a-b7ad-8bb073b206cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:14.726574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c801ea2-ffd3-4e2a-b7ad-8bb073b206cf service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:14.726858 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c801ea2-ffd3-4e2a-b7ad-8bb073b206cf service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:14.727135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c801ea2-ffd3-4e2a-b7ad-8bb073b206cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:14.735969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c5392-854b-4488-9992-fbee5cfe6b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:14.737456 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c5392-854b-4488-9992-fbee5cfe6b0d service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:14.760090 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c801ea2-ffd3-4e2a-b7ad-8bb073b206cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:14.762142 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:14.762276 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c801ea2-ffd3-4e2a-b7ad-8bb073b206cf service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:14.774398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 847/1672] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:14.780842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c5392-854b-4488-9992-fbee5cfe6b0d service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:14.789477 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 826/1673] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:14.804293 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c5392-854b-4488-9992-fbee5cfe6b0d service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:14.821342 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fe684-a796-4417-a5d8-450741ec8a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:14.826223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fe684-a796-4417-a5d8-450741ec8a89 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:14.826515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fe684-a796-4417-a5d8-450741ec8a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:14.826786 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fe684-a796-4417-a5d8-450741ec8a89 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:14.827565 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421fe684-a796-4417-a5d8-450741ec8a89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:14.829746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421fe684-a796-4417-a5d8-450741ec8a89 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:14.861293 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 848/1674] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:14.865606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c801ea2-ffd3-4e2a-b7ad-8bb073b206cf service nova] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:14.884452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c801ea2-ffd3-4e2a-b7ad-8bb073b206cf service nova] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:14.897330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb2a0d-8693-4664-823f-c1b3bdd352f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:14.898379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb2a0d-8693-4664-823f-c1b3bdd352f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:14.898670 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb2a0d-8693-4664-823f-c1b3bdd352f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:14.898953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb2a0d-8693-4664-823f-c1b3bdd352f3 admin admin] PATH_INFO: `/v3/projects/24f8a5d6bc38496792b1fa1562a159ab/users/4465844651ec4ddaa6a772b1aca23db2/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:14.942077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb2a0d-8693-4664-823f-c1b3bdd352f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24f8a5d6bc38496792b1fa1562a159ab, user_id=4465844651ec4ddaa6a772b1aca23db2, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:14.949179 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb2a0d-8693-4664-823f-c1b3bdd352f3 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:14.989758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fe684-a796-4417-a5d8-450741ec8a89 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:14.990406 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 827/1675] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:14 2022] PUT /identity/v3/projects/24f8a5d6bc38496792b1fa1562a159ab/users/4465844651ec4ddaa6a772b1aca23db2/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:14.992071 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-421fe684-a796-4417-a5d8-450741ec8a89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:15.002368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5d4af-ba00-4461-a50a-e8847cd61283 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:15.009656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5d4af-ba00-4461-a50a-e8847cd61283 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:15.009827 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5d4af-ba00-4461-a50a-e8847cd61283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:15.041679 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fe684-a796-4417-a5d8-450741ec8a89 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:15.084300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-afe5d4af-ba00-4461-a50a-e8847cd61283 None None] MFA Rules not processed for user `4465844651ec4ddaa6a772b1aca23db2`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:15.085790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-afe5d4af-ba00-4461-a50a-e8847cd61283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:15.202588 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5432bf-dfee-4098-85c8-38916be85dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:15.203775 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5432bf-dfee-4098-85c8-38916be85dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:15.204061 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5432bf-dfee-4098-85c8-38916be85dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:15.206396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5432bf-dfee-4098-85c8-38916be85dfc admin admin] PATH_INFO: `/v3/projects/57a9df2fc2d44e99b49cbb97ba77ba74` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:15.210642 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5432bf-dfee-4098-85c8-38916be85dfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=57a9df2fc2d44e99b49cbb97ba77ba74)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:15.215764 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5432bf-dfee-4098-85c8-38916be85dfc admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:15.264166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 828/1676] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:15.271729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5d4af-ba00-4461-a50a-e8847cd61283 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:15.274112 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-afe5d4af-ba00-4461-a50a-e8847cd61283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:15.326308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5d4af-ba00-4461-a50a-e8847cd61283 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:15.348917 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8f5432bf-dfee-4098-85c8-38916be85dfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57a9df2fc2d44e99b49cbb97ba77ba74', 'request_id': 'req-8f5432bf-dfee-4098-85c8-38916be85dfc'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:15.368793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8f5432bf-dfee-4098-85c8-38916be85dfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57a9df2fc2d44e99b49cbb97ba77ba74', 'request_id': 'req-8f5432bf-dfee-4098-85c8-38916be85dfc'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:15.393170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8f5432bf-dfee-4098-85c8-38916be85dfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57a9df2fc2d44e99b49cbb97ba77ba74', 'request_id': 'req-8f5432bf-dfee-4098-85c8-38916be85dfc'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:15.422285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8f5432bf-dfee-4098-85c8-38916be85dfc admin admin] The token cache is being invalidate because project 57a9df2fc2d44e99b49cbb97ba77ba74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:15.423771 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8f5432bf-dfee-4098-85c8-38916be85dfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:15.425064 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8f5432bf-dfee-4098-85c8-38916be85dfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:15.431607 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 849/1677] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:14 2022] DELETE /identity/v3/projects/57a9df2fc2d44e99b49cbb97ba77ba74 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:15.431607 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5432bf-dfee-4098-85c8-38916be85dfc admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:15.431607 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-8f5432bf-dfee-4098-85c8-38916be85dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:15.483667 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85064bf-858d-4fa8-be8e-59dc208a0cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:15.484561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85064bf-858d-4fa8-be8e-59dc208a0cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:15.486241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85064bf-858d-4fa8-be8e-59dc208a0cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:15.486241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85064bf-858d-4fa8-be8e-59dc208a0cf2 admin admin] PATH_INFO: `/v3/projects/3e28a026296942abb83826065af4a110` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:15.489555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85064bf-858d-4fa8-be8e-59dc208a0cf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e28a026296942abb83826065af4a110)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:15.490931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: WARNING py.warnings [None req-d85064bf-858d-4fa8-be8e-59dc208a0cf2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 22 11:45:15.490931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: warnings.warn(msg) Jun 22 11:45:15.490931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: Jun 22 11:45:15.495619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85064bf-858d-4fa8-be8e-59dc208a0cf2 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:15.581100 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5432bf-dfee-4098-85c8-38916be85dfc admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:15.594144 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89f704-58b1-44cb-9018-9b83a90406a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:15.595192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89f704-58b1-44cb-9018-9b83a90406a5 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:15.595551 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89f704-58b1-44cb-9018-9b83a90406a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:15.595884 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee89f704-58b1-44cb-9018-9b83a90406a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:15.597679 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee89f704-58b1-44cb-9018-9b83a90406a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:15.602636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee89f704-58b1-44cb-9018-9b83a90406a5 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:15.634490 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d85064bf-858d-4fa8-be8e-59dc208a0cf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e28a026296942abb83826065af4a110', 'request_id': 'req-d85064bf-858d-4fa8-be8e-59dc208a0cf2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:15.648565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d85064bf-858d-4fa8-be8e-59dc208a0cf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e28a026296942abb83826065af4a110', 'request_id': 'req-d85064bf-858d-4fa8-be8e-59dc208a0cf2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:15.650330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:15.667041 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 850/1678] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:15 2022] POST /identity/v3/projects => generated 371 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:15.668101 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89f704-58b1-44cb-9018-9b83a90406a5 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:15.671438 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ee89f704-58b1-44cb-9018-9b83a90406a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:15.679578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:15.686271 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d85064bf-858d-4fa8-be8e-59dc208a0cf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e28a026296942abb83826065af4a110', 'request_id': 'req-d85064bf-858d-4fa8-be8e-59dc208a0cf2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:15.715856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d85064bf-858d-4fa8-be8e-59dc208a0cf2 admin admin] The token cache is being invalidate because project 3e28a026296942abb83826065af4a110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:15.716740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d85064bf-858d-4fa8-be8e-59dc208a0cf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:15.718977 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d85064bf-858d-4fa8-be8e-59dc208a0cf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:15.722843 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 829/1679] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:15 2022] DELETE /identity/v3/projects/3e28a026296942abb83826065af4a110 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:15.729235 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85064bf-858d-4fa8-be8e-59dc208a0cf2 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:15.732939 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d85064bf-858d-4fa8-be8e-59dc208a0cf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:15.830869 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee89f704-58b1-44cb-9018-9b83a90406a5 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:15.842173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3288b-842d-40da-a9ed-9066e33c684a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:15.844221 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3288b-842d-40da-a9ed-9066e33c684a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:15.845944 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3288b-842d-40da-a9ed-9066e33c684a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:15.846255 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3288b-842d-40da-a9ed-9066e33c684a admin admin] PATH_INFO: `/v3/users/0c66b16388734ae98ad2b16104da0bb7` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:15.849683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f3288b-842d-40da-a9ed-9066e33c684a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c66b16388734ae98ad2b16104da0bb7)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:15.852425 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f3288b-842d-40da-a9ed-9066e33c684a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:15.890111 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85064bf-858d-4fa8-be8e-59dc208a0cf2 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:15.903429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288e83d-0f8e-4257-a1a8-0f822449b2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:15.904489 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288e83d-0f8e-4257-a1a8-0f822449b2b2 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:15.905534 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288e83d-0f8e-4257-a1a8-0f822449b2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:15.905999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288e83d-0f8e-4257-a1a8-0f822449b2b2 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:15.909578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e288e83d-0f8e-4257-a1a8-0f822449b2b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:15.912069 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e288e83d-0f8e-4257-a1a8-0f822449b2b2 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:15.949903 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d2f3288b-842d-40da-a9ed-9066e33c684a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c66b16388734ae98ad2b16104da0bb7'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:15.964709 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 830/1680] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:15 2022] POST /identity/v3/users => generated 292 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:15.967778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288e83d-0f8e-4257-a1a8-0f822449b2b2 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:15.974121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e288e83d-0f8e-4257-a1a8-0f822449b2b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:15.976566 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d2f3288b-842d-40da-a9ed-9066e33c684a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c66b16388734ae98ad2b16104da0bb7'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:15.994406 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288e83d-0f8e-4257-a1a8-0f822449b2b2 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:15.996473 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d2f3288b-842d-40da-a9ed-9066e33c684a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c66b16388734ae98ad2b16104da0bb7'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:16.000194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696e9a3-58bd-4e02-80d4-a30c4d011e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:16.001963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696e9a3-58bd-4e02-80d4-a30c4d011e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:16.002298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696e9a3-58bd-4e02-80d4-a30c4d011e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:16.002667 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696e9a3-58bd-4e02-80d4-a30c4d011e1e admin admin] PATH_INFO: `/v3/users/2781d98232fc4a90bce29a7d5ba5b9ed` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:16.004179 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696e9a3-58bd-4e02-80d4-a30c4d011e1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2781d98232fc4a90bce29a7d5ba5b9ed)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:16.006766 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696e9a3-58bd-4e02-80d4-a30c4d011e1e admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:16.011950 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d2f3288b-842d-40da-a9ed-9066e33c684a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c66b16388734ae98ad2b16104da0bb7'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:16.013991 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d2f3288b-842d-40da-a9ed-9066e33c684a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c66b16388734ae98ad2b16104da0bb7'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:16.035153 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 851/1681] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:15 2022] DELETE /identity/v3/users/0c66b16388734ae98ad2b16104da0bb7 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:16.036236 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3288b-842d-40da-a9ed-9066e33c684a admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:16.040472 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d2f3288b-842d-40da-a9ed-9066e33c684a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:16.081252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8696e9a3-58bd-4e02-80d4-a30c4d011e1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2781d98232fc4a90bce29a7d5ba5b9ed'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:16.099126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8696e9a3-58bd-4e02-80d4-a30c4d011e1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2781d98232fc4a90bce29a7d5ba5b9ed'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:16.117372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8696e9a3-58bd-4e02-80d4-a30c4d011e1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2781d98232fc4a90bce29a7d5ba5b9ed'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:16.120165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8696e9a3-58bd-4e02-80d4-a30c4d011e1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2781d98232fc4a90bce29a7d5ba5b9ed'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:16.146035 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8696e9a3-58bd-4e02-80d4-a30c4d011e1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2781d98232fc4a90bce29a7d5ba5b9ed'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:16.166373 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 831/1682] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:15 2022] DELETE /identity/v3/users/2781d98232fc4a90bce29a7d5ba5b9ed => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:16.196517 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3288b-842d-40da-a9ed-9066e33c684a admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:16.268703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:16.325503 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4eaf69-a1db-43ef-9a01-6c5651c5e8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:16.326209 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4eaf69-a1db-43ef-9a01-6c5651c5e8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:16.326496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4eaf69-a1db-43ef-9a01-6c5651c5e8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:16.326774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4eaf69-a1db-43ef-9a01-6c5651c5e8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:16.327546 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4eaf69-a1db-43ef-9a01-6c5651c5e8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:16.329766 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4eaf69-a1db-43ef-9a01-6c5651c5e8e7 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:16.353614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696e9a3-58bd-4e02-80d4-a30c4d011e1e admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:16.354507 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 852/1683] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:16.355228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-8696e9a3-58bd-4e02-80d4-a30c4d011e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:16.384097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696e9a3-58bd-4e02-80d4-a30c4d011e1e admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:16.391589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190f620-6cdb-4ca1-96b1-71f90b1f1929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:16.393640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190f620-6cdb-4ca1-96b1-71f90b1f1929 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:16.394550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190f620-6cdb-4ca1-96b1-71f90b1f1929 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:16.395147 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190f620-6cdb-4ca1-96b1-71f90b1f1929 admin admin] PATH_INFO: `/v3/projects/e6342a74f30a4a63a611697289ebf5b2/users/df50a7bd2c3345afa62265df051b8f2f/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:16.425036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190f620-6cdb-4ca1-96b1-71f90b1f1929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6342a74f30a4a63a611697289ebf5b2, user_id=df50a7bd2c3345afa62265df051b8f2f, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:16.429530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190f620-6cdb-4ca1-96b1-71f90b1f1929 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:16.468438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 832/1684] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:16 2022] PUT /identity/v3/projects/e6342a74f30a4a63a611697289ebf5b2/users/df50a7bd2c3345afa62265df051b8f2f/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:45:16.471167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4eaf69-a1db-43ef-9a01-6c5651c5e8e7 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:16.495034 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4eaf69-a1db-43ef-9a01-6c5651c5e8e7 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:16.691435 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc60ad8-113d-4f53-8f9b-6c2639aa18bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:16.694892 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc60ad8-113d-4f53-8f9b-6c2639aa18bb admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:16.695756 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc60ad8-113d-4f53-8f9b-6c2639aa18bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:16.696751 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc60ad8-113d-4f53-8f9b-6c2639aa18bb admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:16.698290 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc60ad8-113d-4f53-8f9b-6c2639aa18bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:16.703802 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc60ad8-113d-4f53-8f9b-6c2639aa18bb admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:16.756242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 853/1685] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:16.760441 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190f620-6cdb-4ca1-96b1-71f90b1f1929 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:16.780499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190f620-6cdb-4ca1-96b1-71f90b1f1929 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:16.795505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652152d-0d6c-4389-9b2b-3ab438e713a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:16.796438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652152d-0d6c-4389-9b2b-3ab438e713a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:16.796817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652152d-0d6c-4389-9b2b-3ab438e713a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:16.797121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652152d-0d6c-4389-9b2b-3ab438e713a4 admin admin] PATH_INFO: `/v3/projects/e6342a74f30a4a63a611697289ebf5b2/users/df50a7bd2c3345afa62265df051b8f2f/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:16.800450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652152d-0d6c-4389-9b2b-3ab438e713a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6342a74f30a4a63a611697289ebf5b2, user_id=df50a7bd2c3345afa62265df051b8f2f, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:16.808124 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652152d-0d6c-4389-9b2b-3ab438e713a4 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:16.836823 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 833/1686] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:16 2022] PUT /identity/v3/projects/e6342a74f30a4a63a611697289ebf5b2/users/df50a7bd2c3345afa62265df051b8f2f/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:16.847611 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d83fba-c12a-4e98-8dfd-2ffba8a82ed1 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:16.847910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d83fba-c12a-4e98-8dfd-2ffba8a82ed1 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:16.848190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d83fba-c12a-4e98-8dfd-2ffba8a82ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:16.885458 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b9ea4-e203-4fac-9ad2-a04197a0be52 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:16.885796 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b9ea4-e203-4fac-9ad2-a04197a0be52 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:16.886819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b9ea4-e203-4fac-9ad2-a04197a0be52 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:16.890575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 834/1687] 10.222.0.36 () {54 vars in 912 bytes} [Wed Jun 22 11:45:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:45:16.897122 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac04a0-fe39-4e8e-9e6b-8593c8a6f26d None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:16.898718 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac04a0-fe39-4e8e-9e6b-8593c8a6f26d None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:16.899052 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac04a0-fe39-4e8e-9e6b-8593c8a6f26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:16.949439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=82,keystone:DELETE=12,keystone:INSERT=6 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:16.973603 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-22d83fba-c12a-4e98-8dfd-2ffba8a82ed1 None None] MFA Rules not processed for user `df50a7bd2c3345afa62265df051b8f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:16.975146 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-22d83fba-c12a-4e98-8dfd-2ffba8a82ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:16.990390 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-3fac04a0-fe39-4e8e-9e6b-8593c8a6f26d None None] MFA Rules not processed for user `c00d0bb777a044a3a4d4540d187d938a`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:16.991829 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-3fac04a0-fe39-4e8e-9e6b-8593c8a6f26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:17.119061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 835/1688] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jun 22 11:45:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:45:17.120131 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac04a0-fe39-4e8e-9e6b-8593c8a6f26d None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:17.121503 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-3fac04a0-fe39-4e8e-9e6b-8593c8a6f26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:17.160988 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 854/1689] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:16 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:17.162629 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45aae13-1078-45c1-a51d-ddbf644ff899 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:17.163025 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45aae13-1078-45c1-a51d-ddbf644ff899 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:17.163536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45aae13-1078-45c1-a51d-ddbf644ff899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:17.222808 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-b45aae13-1078-45c1-a51d-ddbf644ff899 None None] MFA Rules not processed for user `4d7c1adbb2ad456eaf5793a15d6fcb4b`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:17.230640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b45aae13-1078-45c1-a51d-ddbf644ff899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:17.316974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:17.319709 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac04a0-fe39-4e8e-9e6b-8593c8a6f26d None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:17.333067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7c493-4332-420d-b8b6-fa3d8986be5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:17.338010 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7c493-4332-420d-b8b6-fa3d8986be5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:17.338376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7c493-4332-420d-b8b6-fa3d8986be5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:17.338597 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7c493-4332-420d-b8b6-fa3d8986be5b admin admin] PATH_INFO: `/v3/projects/2889a1aa2fa64d6c8d83a9893cdd7b21` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:17.340717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7c493-4332-420d-b8b6-fa3d8986be5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2889a1aa2fa64d6c8d83a9893cdd7b21)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:17.344293 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7c493-4332-420d-b8b6-fa3d8986be5b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:17.367320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 855/1690] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:17.367320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45aae13-1078-45c1-a51d-ddbf644ff899 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:17.367320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b45aae13-1078-45c1-a51d-ddbf644ff899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:17.379697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45aae13-1078-45c1-a51d-ddbf644ff899 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:17.385115 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf18661-fb3a-48e0-96ac-0d9b8360ac50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:17.385756 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf18661-fb3a-48e0-96ac-0d9b8360ac50 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:17.386021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf18661-fb3a-48e0-96ac-0d9b8360ac50 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:17.386302 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf18661-fb3a-48e0-96ac-0d9b8360ac50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:17.395743 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf18661-fb3a-48e0-96ac-0d9b8360ac50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:17.397714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf18661-fb3a-48e0-96ac-0d9b8360ac50 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:17.409555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 856/1691] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:17.410580 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf18661-fb3a-48e0-96ac-0d9b8360ac50 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:17.412931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-faf18661-fb3a-48e0-96ac-0d9b8360ac50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:17.504228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-12c7c493-4332-420d-b8b6-fa3d8986be5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2889a1aa2fa64d6c8d83a9893cdd7b21', 'request_id': 'req-12c7c493-4332-420d-b8b6-fa3d8986be5b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:17.540362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-12c7c493-4332-420d-b8b6-fa3d8986be5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2889a1aa2fa64d6c8d83a9893cdd7b21', 'request_id': 'req-12c7c493-4332-420d-b8b6-fa3d8986be5b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:17.556118 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-12c7c493-4332-420d-b8b6-fa3d8986be5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2889a1aa2fa64d6c8d83a9893cdd7b21', 'request_id': 'req-12c7c493-4332-420d-b8b6-fa3d8986be5b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:17.586138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-12c7c493-4332-420d-b8b6-fa3d8986be5b admin admin] The token cache is being invalidate because project 2889a1aa2fa64d6c8d83a9893cdd7b21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:17.586522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-12c7c493-4332-420d-b8b6-fa3d8986be5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:17.591934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-12c7c493-4332-420d-b8b6-fa3d8986be5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:17.593766 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 836/1692] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:17 2022] DELETE /identity/v3/projects/2889a1aa2fa64d6c8d83a9893cdd7b21 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:17.598367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7c493-4332-420d-b8b6-fa3d8986be5b admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:17.601039 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-12c7c493-4332-420d-b8b6-fa3d8986be5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:17.667678 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf18661-fb3a-48e0-96ac-0d9b8360ac50 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:17.687419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:17.750541 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7c493-4332-420d-b8b6-fa3d8986be5b admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:17.765016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe5283-8766-49b3-8abc-1bf5165e4911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:17.766578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe5283-8766-49b3-8abc-1bf5165e4911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:17.769188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe5283-8766-49b3-8abc-1bf5165e4911 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:17.769613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe5283-8766-49b3-8abc-1bf5165e4911 admin admin] PATH_INFO: `/v3/projects/a4364d831dd84245ac42f4f461f74c74` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:17.774594 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe5283-8766-49b3-8abc-1bf5165e4911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4364d831dd84245ac42f4f461f74c74)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:17.778390 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe5283-8766-49b3-8abc-1bf5165e4911 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:17.805578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f61ca-5b94-429b-a870-409c5ce840da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:17.806548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f61ca-5b94-429b-a870-409c5ce840da service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:17.806841 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f61ca-5b94-429b-a870-409c5ce840da service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:17.807118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f61ca-5b94-429b-a870-409c5ce840da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:17.813197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e20f61ca-5b94-429b-a870-409c5ce840da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:17.903871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a2fe5283-8766-49b3-8abc-1bf5165e4911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4364d831dd84245ac42f4f461f74c74', 'request_id': 'req-a2fe5283-8766-49b3-8abc-1bf5165e4911'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:17.920562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a2fe5283-8766-49b3-8abc-1bf5165e4911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4364d831dd84245ac42f4f461f74c74', 'request_id': 'req-a2fe5283-8766-49b3-8abc-1bf5165e4911'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:17.945610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a2fe5283-8766-49b3-8abc-1bf5165e4911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4364d831dd84245ac42f4f461f74c74', 'request_id': 'req-a2fe5283-8766-49b3-8abc-1bf5165e4911'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:17.968859 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a2fe5283-8766-49b3-8abc-1bf5165e4911 admin admin] The token cache is being invalidate because project a4364d831dd84245ac42f4f461f74c74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:17.969057 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a2fe5283-8766-49b3-8abc-1bf5165e4911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:17.971246 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a2fe5283-8766-49b3-8abc-1bf5165e4911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:17.973617 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 837/1693] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:17 2022] DELETE /identity/v3/projects/a4364d831dd84245ac42f4f461f74c74 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:17.979972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe5283-8766-49b3-8abc-1bf5165e4911 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:17.982934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a2fe5283-8766-49b3-8abc-1bf5165e4911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:18.000721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f61ca-5b94-429b-a870-409c5ce840da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:18.004300 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f61ca-5b94-429b-a870-409c5ce840da service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:18.030605 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 857/1694] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:17 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:18.035271 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af0f0c-52bc-4007-adbc-dbd2c4d15c1b None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:18.035582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af0f0c-52bc-4007-adbc-dbd2c4d15c1b None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:18.035861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af0f0c-52bc-4007-adbc-dbd2c4d15c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:18.069275 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-22af0f0c-52bc-4007-adbc-dbd2c4d15c1b None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:18.077663 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-22af0f0c-52bc-4007-adbc-dbd2c4d15c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:18.150723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:18.205580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe5283-8766-49b3-8abc-1bf5165e4911 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:18.216525 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0845b-840f-4636-9780-f57c8f5963e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:18.217459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0845b-840f-4636-9780-f57c8f5963e3 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:18.217650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0845b-840f-4636-9780-f57c8f5963e3 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:18.218274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0845b-840f-4636-9780-f57c8f5963e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:18.222772 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-cea0845b-840f-4636-9780-f57c8f5963e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:18.260545 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 858/1695] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:18.265933 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d7e98-c79e-43f7-9f9e-d7c357221883 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:18.266259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d7e98-c79e-43f7-9f9e-d7c357221883 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:18.266631 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d7e98-c79e-43f7-9f9e-d7c357221883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:18.312915 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-dd9d7e98-c79e-43f7-9f9e-d7c357221883 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:18.317851 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-dd9d7e98-c79e-43f7-9f9e-d7c357221883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:18.340477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 859/1696] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:18.348620 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d7e98-c79e-43f7-9f9e-d7c357221883 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:18.351255 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-dd9d7e98-c79e-43f7-9f9e-d7c357221883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:18.372199 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0845b-840f-4636-9780-f57c8f5963e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:18.374879 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0845b-840f-4636-9780-f57c8f5963e3 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:18.386181 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 838/1697] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:18.387742 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1347d4-1f83-44fd-8408-867355fd6694 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:18.388041 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1347d4-1f83-44fd-8408-867355fd6694 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:18.388320 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1347d4-1f83-44fd-8408-867355fd6694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:18.396487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:18.422558 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-2c1347d4-1f83-44fd-8408-867355fd6694 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:18.427451 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2c1347d4-1f83-44fd-8408-867355fd6694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:18.466908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 839/1698] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:18.468322 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1347d4-1f83-44fd-8408-867355fd6694 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:18.470469 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2c1347d4-1f83-44fd-8408-867355fd6694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:18.492840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d7e98-c79e-43f7-9f9e-d7c357221883 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:18.500376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1347d4-1f83-44fd-8408-867355fd6694 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:18.506007 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d45b42-2768-45dd-af8b-cc9d4eea6295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:18.507036 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d45b42-2768-45dd-af8b-cc9d4eea6295 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:18.507329 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d45b42-2768-45dd-af8b-cc9d4eea6295 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:18.507647 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d45b42-2768-45dd-af8b-cc9d4eea6295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:18.511881 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e7628-4a3c-49f3-bbd0-30f790751eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:18.512854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e7628-4a3c-49f3-bbd0-30f790751eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:18.513051 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e7628-4a3c-49f3-bbd0-30f790751eef admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:18.513322 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e7628-4a3c-49f3-bbd0-30f790751eef admin admin] PATH_INFO: `/v3/projects/4689468bb3224ac68a6e3e6b4067b947` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:18.513808 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-62d45b42-2768-45dd-af8b-cc9d4eea6295 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:18.518482 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e7628-4a3c-49f3-bbd0-30f790751eef admin admin] RBAC: Authorizing `identity:delete_project(project_id=4689468bb3224ac68a6e3e6b4067b947)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:18.520759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e7628-4a3c-49f3-bbd0-30f790751eef admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:18.700814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ed0e7628-4a3c-49f3-bbd0-30f790751eef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4689468bb3224ac68a6e3e6b4067b947', 'request_id': 'req-ed0e7628-4a3c-49f3-bbd0-30f790751eef'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:18.713129 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ed0e7628-4a3c-49f3-bbd0-30f790751eef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4689468bb3224ac68a6e3e6b4067b947', 'request_id': 'req-ed0e7628-4a3c-49f3-bbd0-30f790751eef'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:18.722873 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ed0e7628-4a3c-49f3-bbd0-30f790751eef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4689468bb3224ac68a6e3e6b4067b947', 'request_id': 'req-ed0e7628-4a3c-49f3-bbd0-30f790751eef'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:18.738942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ed0e7628-4a3c-49f3-bbd0-30f790751eef admin admin] The token cache is being invalidate because project 4689468bb3224ac68a6e3e6b4067b947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:18.739493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ed0e7628-4a3c-49f3-bbd0-30f790751eef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:18.740110 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ed0e7628-4a3c-49f3-bbd0-30f790751eef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:18.746580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 840/1699] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:18 2022] DELETE /identity/v3/projects/4689468bb3224ac68a6e3e6b4067b947 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:18.748032 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb97cc-2ca8-4e34-b736-6e0e4215cc8f None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:18.748427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb97cc-2ca8-4e34-b736-6e0e4215cc8f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:18.748717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb97cc-2ca8-4e34-b736-6e0e4215cc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:18.759858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d45b42-2768-45dd-af8b-cc9d4eea6295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:18.762379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d45b42-2768-45dd-af8b-cc9d4eea6295 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:18.776865 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 860/1700] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:18.777798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-effb97cc-2ca8-4e34-b736-6e0e4215cc8f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:18.779230 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-effb97cc-2ca8-4e34-b736-6e0e4215cc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:18.779486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d45b42-2768-45dd-af8b-cc9d4eea6295 service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:18.814944 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d45b42-2768-45dd-af8b-cc9d4eea6295 service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:18.937022 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 841/1701] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:18.946234 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b2be-5e8e-4235-96aa-4ce61cbf56db None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:18.948638 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b2be-5e8e-4235-96aa-4ce61cbf56db None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:18.948926 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b2be-5e8e-4235-96aa-4ce61cbf56db None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:18.957339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 842/1702] 10.222.0.36 () {54 vars in 960 bytes} [Wed Jun 22 11:45:18 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:45:18.960933 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70213ae-0bd8-4551-80e9-bab0316e7d7e None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:18.962928 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70213ae-0bd8-4551-80e9-bab0316e7d7e None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:18.963381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70213ae-0bd8-4551-80e9-bab0316e7d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:18.976121 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbdd7db-4eae-446d-b52a-cb59eb0e962e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:18.976871 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdd7db-4eae-446d-b52a-cb59eb0e962e admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:18.977153 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdd7db-4eae-446d-b52a-cb59eb0e962e admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:18.977425 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdd7db-4eae-446d-b52a-cb59eb0e962e admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:18.978222 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbdd7db-4eae-446d-b52a-cb59eb0e962e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:18.979383 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbdd7db-4eae-446d-b52a-cb59eb0e962e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:18.981131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbdd7db-4eae-446d-b52a-cb59eb0e962e admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:19.004106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-d70213ae-0bd8-4551-80e9-bab0316e7d7e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:19.007846 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d70213ae-0bd8-4551-80e9-bab0316e7d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:19.028727 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 861/1703] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:19.030367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbdd7db-4eae-446d-b52a-cb59eb0e962e admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:19.039732 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 843/1704] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:19.042237 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3724d-7fbd-4855-8216-01d459e97188 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:19.042766 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3724d-7fbd-4855-8216-01d459e97188 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:19.043449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3724d-7fbd-4855-8216-01d459e97188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:19.046436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbdd7db-4eae-446d-b52a-cb59eb0e962e admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:19.054163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf2b72-07c3-49c8-97b2-34887de7ea00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:19.055900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf2b72-07c3-49c8-97b2-34887de7ea00 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:19.056504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf2b72-07c3-49c8-97b2-34887de7ea00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:19.057357 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf2b72-07c3-49c8-97b2-34887de7ea00 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:19.058381 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf2b72-07c3-49c8-97b2-34887de7ea00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:19.058875 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf2b72-07c3-49c8-97b2-34887de7ea00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:19.060855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf2b72-07c3-49c8-97b2-34887de7ea00 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:19.076862 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:19.098925 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 862/1705] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:19.100198 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf2b72-07c3-49c8-97b2-34887de7ea00 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:19.102385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-2faf2b72-07c3-49c8-97b2-34887de7ea00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:19.108262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-bee3724d-7fbd-4855-8216-01d459e97188 None None] MFA Rules not processed for user `93630ed5e6dc4a5f958bc7cf1a68a26c`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:19.111798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-bee3724d-7fbd-4855-8216-01d459e97188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:19.119672 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf2b72-07c3-49c8-97b2-34887de7ea00 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:19.125288 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c423d3-c194-4aea-a1b8-32fedafdb556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:19.126557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c423d3-c194-4aea-a1b8-32fedafdb556 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:19.126843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c423d3-c194-4aea-a1b8-32fedafdb556 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:19.127120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c423d3-c194-4aea-a1b8-32fedafdb556 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:19.127901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c423d3-c194-4aea-a1b8-32fedafdb556 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:19.130551 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c423d3-c194-4aea-a1b8-32fedafdb556 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:19.201391 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 863/1706] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:19 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:19.205504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85669d22-7550-46b9-9548-a1c260f7db95 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:19.210988 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85669d22-7550-46b9-9548-a1c260f7db95 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:19.211640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85669d22-7550-46b9-9548-a1c260f7db95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:19.269444 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-85669d22-7550-46b9-9548-a1c260f7db95 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:19.270795 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-85669d22-7550-46b9-9548-a1c260f7db95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:19.348460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 844/1707] 10.222.0.36 () {58 vars in 1057 bytes} [Wed Jun 22 11:45:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:19.353348 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3724d-7fbd-4855-8216-01d459e97188 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:19.385924 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3724d-7fbd-4855-8216-01d459e97188 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:19.393603 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:19.429902 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 864/1708] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:19.433271 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85669d22-7550-46b9-9548-a1c260f7db95 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:19.457895 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85669d22-7550-46b9-9548-a1c260f7db95 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:19.466082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1980e-c44b-4836-81c0-3d447add360a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:19.467110 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1980e-c44b-4836-81c0-3d447add360a admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:19.467679 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1980e-c44b-4836-81c0-3d447add360a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:19.468501 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1980e-c44b-4836-81c0-3d447add360a admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:19.469428 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1980e-c44b-4836-81c0-3d447add360a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:19.472389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1980e-c44b-4836-81c0-3d447add360a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:19.501463 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 865/1709] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:19 2022] POST /identity/v3/users => generated 285 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:19.503483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1980e-c44b-4836-81c0-3d447add360a admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:19.555265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63cd20-e78d-48d7-b1fe-2b477f9fa408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:19.556007 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63cd20-e78d-48d7-b1fe-2b477f9fa408 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:19.556288 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63cd20-e78d-48d7-b1fe-2b477f9fa408 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:19.556576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63cd20-e78d-48d7-b1fe-2b477f9fa408 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:19.557290 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63cd20-e78d-48d7-b1fe-2b477f9fa408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:19.563429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63cd20-e78d-48d7-b1fe-2b477f9fa408 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:19.638915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 845/1710] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:19 2022] POST /identity/v3/projects => generated 385 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:19.638915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63cd20-e78d-48d7-b1fe-2b477f9fa408 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:19.661028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1980e-c44b-4836-81c0-3d447add360a admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:19.784479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63cd20-e78d-48d7-b1fe-2b477f9fa408 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:19.794324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b077f-ec54-4a53-bc40-26e66a14e264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:19.795500 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b077f-ec54-4a53-bc40-26e66a14e264 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:19.796094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b077f-ec54-4a53-bc40-26e66a14e264 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:19.796844 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b077f-ec54-4a53-bc40-26e66a14e264 admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:19.798930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23bd81-be60-4641-9edb-07e5fef68b5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93630ed5e6dc4a5f958bc7cf1a68a26c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:19.799402 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b077f-ec54-4a53-bc40-26e66a14e264 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:19.800063 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b077f-ec54-4a53-bc40-26e66a14e264 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:19.803730 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b077f-ec54-4a53-bc40-26e66a14e264 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:19.810055 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23bd81-be60-4641-9edb-07e5fef68b5d service placement] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:19.810880 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23bd81-be60-4641-9edb-07e5fef68b5d service placement] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:19.811341 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23bd81-be60-4641-9edb-07e5fef68b5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:19.816194 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-cc23bd81-be60-4641-9edb-07e5fef68b5d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:19.851913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 846/1711] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:19.851913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b077f-ec54-4a53-bc40-26e66a14e264 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:19.857097 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:19.861653 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23bd81-be60-4641-9edb-07e5fef68b5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:19.864761 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23bd81-be60-4641-9edb-07e5fef68b5d service placement] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:19.868703 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b077f-ec54-4a53-bc40-26e66a14e264 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:19.878608 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144bed2-51cb-4846-9d37-615ea0e91517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:19.879310 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144bed2-51cb-4846-9d37-615ea0e91517 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:19.879673 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144bed2-51cb-4846-9d37-615ea0e91517 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:19.879958 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144bed2-51cb-4846-9d37-615ea0e91517 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:19.880967 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144bed2-51cb-4846-9d37-615ea0e91517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:19.882718 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144bed2-51cb-4846-9d37-615ea0e91517 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:19.885310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 866/1712] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:19.888109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23bd81-be60-4641-9edb-07e5fef68b5d service placement] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:19.917965 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23bd81-be60-4641-9edb-07e5fef68b5d service placement] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:19.920069 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 847/1713] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:19.927833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971c5d3-a26d-4092-9226-9d158fc068e3 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:19.928129 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971c5d3-a26d-4092-9226-9d158fc068e3 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:19.928414 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971c5d3-a26d-4092-9226-9d158fc068e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:19.934771 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e12856-2dba-4d68-a544-8c018085a8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:19.935488 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e12856-2dba-4d68-a544-8c018085a8cd admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:19.936312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e12856-2dba-4d68-a544-8c018085a8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:19.936868 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e12856-2dba-4d68-a544-8c018085a8cd admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:19.941488 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e12856-2dba-4d68-a544-8c018085a8cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:19.943454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e12856-2dba-4d68-a544-8c018085a8cd admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:19.993601 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 867/1714] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:19 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:19.996315 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e12856-2dba-4d68-a544-8c018085a8cd admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:20.002246 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-5971c5d3-a26d-4092-9226-9d158fc068e3 None None] MFA Rules not processed for user `27cc3d38926f470c91eeb6aecfc86a48`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:20.003710 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-5971c5d3-a26d-4092-9226-9d158fc068e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:20.023209 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e12856-2dba-4d68-a544-8c018085a8cd admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:20.049512 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543d32e-745d-4d99-ab16-ad9f097966dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:20.050426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543d32e-745d-4d99-ab16-ad9f097966dc admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:20.050783 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543d32e-745d-4d99-ab16-ad9f097966dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:20.052493 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543d32e-745d-4d99-ab16-ad9f097966dc admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:20.054407 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543d32e-745d-4d99-ab16-ad9f097966dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:20.059791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543d32e-745d-4d99-ab16-ad9f097966dc admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:20.135553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 868/1715] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:19 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:20.136655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543d32e-745d-4d99-ab16-ad9f097966dc admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:20.195479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 848/1716] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:20.195479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971c5d3-a26d-4092-9226-9d158fc068e3 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:20.236965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971c5d3-a26d-4092-9226-9d158fc068e3 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:20.252295 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:20.348158 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543d32e-745d-4d99-ab16-ad9f097966dc admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:20.364860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895c787-b926-40c3-8f00-418f38a4d214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:20.365563 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895c787-b926-40c3-8f00-418f38a4d214 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:20.365849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895c787-b926-40c3-8f00-418f38a4d214 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:20.366127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895c787-b926-40c3-8f00-418f38a4d214 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:20.369258 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895c787-b926-40c3-8f00-418f38a4d214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:20.372738 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895c787-b926-40c3-8f00-418f38a4d214 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:20.403409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 869/1717] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:20.404585 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895c787-b926-40c3-8f00-418f38a4d214 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:20.416399 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec4e82-c3af-4fbc-95de-36e93207911c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:20.417113 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec4e82-c3af-4fbc-95de-36e93207911c admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:20.417400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec4e82-c3af-4fbc-95de-36e93207911c admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:20.417701 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec4e82-c3af-4fbc-95de-36e93207911c admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:20.418431 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec4e82-c3af-4fbc-95de-36e93207911c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:20.420120 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec4e82-c3af-4fbc-95de-36e93207911c admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:20.424868 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895c787-b926-40c3-8f00-418f38a4d214 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:20.430961 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1186d93-327b-4233-a4fa-f19e81f4736f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:20.432551 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1186d93-327b-4233-a4fa-f19e81f4736f admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:20.432843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1186d93-327b-4233-a4fa-f19e81f4736f admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:20.433923 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1186d93-327b-4233-a4fa-f19e81f4736f admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:20.436257 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1186d93-327b-4233-a4fa-f19e81f4736f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:20.438231 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1186d93-327b-4233-a4fa-f19e81f4736f admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:20.453586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 849/1718] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:20.453586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9b8ba-dcc0-4984-b2aa-df52cd496043 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:20.453586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9b8ba-dcc0-4984-b2aa-df52cd496043 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:20.453586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9b8ba-dcc0-4984-b2aa-df52cd496043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:20.486251 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 870/1719] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:20 2022] POST /identity/v3/users => generated 293 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:20.487700 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1186d93-327b-4233-a4fa-f19e81f4736f admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:20.507020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-16f9b8ba-dcc0-4984-b2aa-df52cd496043 None None] MFA Rules not processed for user `27cc3d38926f470c91eeb6aecfc86a48`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:20.509745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-16f9b8ba-dcc0-4984-b2aa-df52cd496043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:20.515303 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1186d93-327b-4233-a4fa-f19e81f4736f admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:20.523808 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b8d2e-d32b-41bd-b34c-86b4ef30fd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:20.525097 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b8d2e-d32b-41bd-b34c-86b4ef30fd8b admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:20.527487 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b8d2e-d32b-41bd-b34c-86b4ef30fd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:20.528158 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b8d2e-d32b-41bd-b34c-86b4ef30fd8b admin admin] PATH_INFO: `/v3/projects/a37fd61419f7409689eb7bc3807bf16c/users/5e74ad4e32e74ebc91bf4daa1bc0dce2/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:20.588581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b8d2e-d32b-41bd-b34c-86b4ef30fd8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a37fd61419f7409689eb7bc3807bf16c, user_id=5e74ad4e32e74ebc91bf4daa1bc0dce2, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:20.600870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b8d2e-d32b-41bd-b34c-86b4ef30fd8b admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:20.638109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 871/1720] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:20 2022] PUT /identity/v3/projects/a37fd61419f7409689eb7bc3807bf16c/users/5e74ad4e32e74ebc91bf4daa1bc0dce2/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:20.638966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b8d2e-d32b-41bd-b34c-86b4ef30fd8b admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:20.643005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 850/1721] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:20.646025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9b8ba-dcc0-4984-b2aa-df52cd496043 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:20.664814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9b8ba-dcc0-4984-b2aa-df52cd496043 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:20.790817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b8d2e-d32b-41bd-b34c-86b4ef30fd8b admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:20.793064 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225f8b7-f32b-46df-8ea0-83ef95e2bf61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:20.794635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225f8b7-f32b-46df-8ea0-83ef95e2bf61 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:20.795508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225f8b7-f32b-46df-8ea0-83ef95e2bf61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:20.795964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225f8b7-f32b-46df-8ea0-83ef95e2bf61 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:20.796862 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225f8b7-f32b-46df-8ea0-83ef95e2bf61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:20.801153 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225f8b7-f32b-46df-8ea0-83ef95e2bf61 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:20.812371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53775f2-b45a-4b2d-a0f2-a2bc8daee8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:20.813467 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53775f2-b45a-4b2d-a0f2-a2bc8daee8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:20.818500 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53775f2-b45a-4b2d-a0f2-a2bc8daee8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:20.819309 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53775f2-b45a-4b2d-a0f2-a2bc8daee8f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:20.820403 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53775f2-b45a-4b2d-a0f2-a2bc8daee8f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:20.826244 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 851/1722] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:20.826244 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225f8b7-f32b-46df-8ea0-83ef95e2bf61 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:20.828058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53775f2-b45a-4b2d-a0f2-a2bc8daee8f4 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:20.832003 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d225f8b7-f32b-46df-8ea0-83ef95e2bf61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:20.865483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 872/1723] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:20.870620 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5e32f-30fb-4f3a-9e05-37709a9c6b2e None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:20.872531 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5e32f-30fb-4f3a-9e05-37709a9c6b2e None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:20.875548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5e32f-30fb-4f3a-9e05-37709a9c6b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:20.897916 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:20.952538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:20.979572 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-1ed5e32f-30fb-4f3a-9e05-37709a9c6b2e None None] MFA Rules not processed for user `5e74ad4e32e74ebc91bf4daa1bc0dce2`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:20.984734 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-1ed5e32f-30fb-4f3a-9e05-37709a9c6b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:21.011266 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225f8b7-f32b-46df-8ea0-83ef95e2bf61 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:21.023877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82715b80-79fa-42a0-aeaf-56ad16112851 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:21.025236 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82715b80-79fa-42a0-aeaf-56ad16112851 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:21.025659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82715b80-79fa-42a0-aeaf-56ad16112851 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:21.026088 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82715b80-79fa-42a0-aeaf-56ad16112851 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:21.184397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 873/1724] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:20 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:21.187715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5e32f-30fb-4f3a-9e05-37709a9c6b2e None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:21.188269 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82715b80-79fa-42a0-aeaf-56ad16112851 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:21.190322 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82715b80-79fa-42a0-aeaf-56ad16112851 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:21.219334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 852/1725] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:21.222810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82715b80-79fa-42a0-aeaf-56ad16112851 service nova] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:21.238112 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5e32f-30fb-4f3a-9e05-37709a9c6b2e None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:21.259526 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f39c6-ccc1-40c7-b066-b96e6174628c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:21.260478 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f39c6-ccc1-40c7-b066-b96e6174628c admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:21.260974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f39c6-ccc1-40c7-b066-b96e6174628c admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:21.262238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f39c6-ccc1-40c7-b066-b96e6174628c admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:21.263603 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f39c6-ccc1-40c7-b066-b96e6174628c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:21.265643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f39c6-ccc1-40c7-b066-b96e6174628c admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:21.274564 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82715b80-79fa-42a0-aeaf-56ad16112851 service nova] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:21.279080 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 874/1726] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:21.280199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f39c6-ccc1-40c7-b066-b96e6174628c admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:21.288136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4b7c7-a19d-40b9-94ed-c61318f99397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:21.289218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4b7c7-a19d-40b9-94ed-c61318f99397 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:21.289646 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4b7c7-a19d-40b9-94ed-c61318f99397 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:21.290228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4b7c7-a19d-40b9-94ed-c61318f99397 admin admin] PATH_INFO: `/v3/projects/c83bc3db4c6148e1b0cd19ad92312947/users/7859085336694721a5cc7a70c8cc0d64/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:21.311893 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f39c6-ccc1-40c7-b066-b96e6174628c admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:21.324184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462187c4-6b66-4ba8-92d0-30ceadb24177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:21.326449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462187c4-6b66-4ba8-92d0-30ceadb24177 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:21.326920 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4b7c7-a19d-40b9-94ed-c61318f99397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c83bc3db4c6148e1b0cd19ad92312947, user_id=7859085336694721a5cc7a70c8cc0d64, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:21.332607 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462187c4-6b66-4ba8-92d0-30ceadb24177 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:21.332952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462187c4-6b66-4ba8-92d0-30ceadb24177 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:21.333697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462187c4-6b66-4ba8-92d0-30ceadb24177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:21.340205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4b7c7-a19d-40b9-94ed-c61318f99397 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:21.341582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462187c4-6b66-4ba8-92d0-30ceadb24177 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:21.436514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 853/1727] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:21 2022] PUT /identity/v3/projects/c83bc3db4c6148e1b0cd19ad92312947/users/7859085336694721a5cc7a70c8cc0d64/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:21.436514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4b7c7-a19d-40b9-94ed-c61318f99397 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:21.507731 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 875/1728] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:21 2022] POST /identity/v3/projects => generated 367 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:21.510110 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1ace6-83ce-4b1f-85f0-6f326d1401c7 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:21.512305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1ace6-83ce-4b1f-85f0-6f326d1401c7 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:21.513814 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1ace6-83ce-4b1f-85f0-6f326d1401c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:21.628654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-24e1ace6-83ce-4b1f-85f0-6f326d1401c7 None None] MFA Rules not processed for user `7859085336694721a5cc7a70c8cc0d64`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:21.631272 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-24e1ace6-83ce-4b1f-85f0-6f326d1401c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:21.650203 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4b7c7-a19d-40b9-94ed-c61318f99397 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:21.812201 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 876/1729] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:21.813975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1ace6-83ce-4b1f-85f0-6f326d1401c7 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:21.827429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9f708-a642-4b85-9c32-078cace8ea63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:21.828444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9f708-a642-4b85-9c32-078cace8ea63 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:21.828813 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9f708-a642-4b85-9c32-078cace8ea63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:21.829769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9f708-a642-4b85-9c32-078cace8ea63 admin admin] PATH_INFO: `/v3/projects/5e577782a02c4e55ab90699dc6401bce/users/d0c7435796d74c6781e4903cf2aedc14/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:21.840916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:21.864067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b9f708-a642-4b85-9c32-078cace8ea63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e577782a02c4e55ab90699dc6401bce, user_id=d0c7435796d74c6781e4903cf2aedc14, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:21.866921 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b9f708-a642-4b85-9c32-078cace8ea63 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:21.898135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 854/1730] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:21 2022] PUT /identity/v3/projects/5e577782a02c4e55ab90699dc6401bce/users/d0c7435796d74c6781e4903cf2aedc14/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:21.898927 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9f708-a642-4b85-9c32-078cace8ea63 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:21.954691 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:21.979308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1ace6-83ce-4b1f-85f0-6f326d1401c7 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:22.025848 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9f708-a642-4b85-9c32-078cace8ea63 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:22.039837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83a2cb-7260-4a8c-b008-6b46e16504ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:22.040537 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83a2cb-7260-4a8c-b008-6b46e16504ac admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:22.040948 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83a2cb-7260-4a8c-b008-6b46e16504ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:22.041283 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83a2cb-7260-4a8c-b008-6b46e16504ac admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:22.041991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83a2cb-7260-4a8c-b008-6b46e16504ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:22.044073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83a2cb-7260-4a8c-b008-6b46e16504ac admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:22.091759 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf863d6a-1ccc-4211-853b-030cc3095403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:22.092652 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf863d6a-1ccc-4211-853b-030cc3095403 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:22.093157 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf863d6a-1ccc-4211-853b-030cc3095403 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:22.093516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf863d6a-1ccc-4211-853b-030cc3095403 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:22.094699 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf863d6a-1ccc-4211-853b-030cc3095403 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:22.099375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf863d6a-1ccc-4211-853b-030cc3095403 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:22.113105 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 855/1731] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:21 2022] POST /identity/v3/projects => generated 383 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:22.116779 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ea27d-f85d-469a-9df7-6460906f6d0c None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:22.117089 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ea27d-f85d-469a-9df7-6460906f6d0c None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:22.117520 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ea27d-f85d-469a-9df7-6460906f6d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:22.174659 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 877/1732] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:21 2022] POST /identity/v3/users => generated 290 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:22.177710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf863d6a-1ccc-4211-853b-030cc3095403 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:22.204160 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf863d6a-1ccc-4211-853b-030cc3095403 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:22.218490 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-ba9ea27d-f85d-469a-9df7-6460906f6d0c None None] MFA Rules not processed for user `d0c7435796d74c6781e4903cf2aedc14`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:22.220397 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ba9ea27d-f85d-469a-9df7-6460906f6d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:22.344385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 856/1733] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:22.348930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6996891-1f89-45d6-b087-5d566041950f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:22.350011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6996891-1f89-45d6-b087-5d566041950f admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:22.350218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ea27d-f85d-469a-9df7-6460906f6d0c None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:22.350619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6996891-1f89-45d6-b087-5d566041950f admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:22.351047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6996891-1f89-45d6-b087-5d566041950f admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:22.351944 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6996891-1f89-45d6-b087-5d566041950f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:22.354098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6996891-1f89-45d6-b087-5d566041950f admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:22.384969 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ea27d-f85d-469a-9df7-6460906f6d0c None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:22.397354 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 878/1734] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:22 2022] POST /identity/v3/users => generated 292 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:22.398909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699c7a9-5c58-432e-b45a-7cbb8bfdbc1a None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:22.399406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699c7a9-5c58-432e-b45a-7cbb8bfdbc1a None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:22.400269 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699c7a9-5c58-432e-b45a-7cbb8bfdbc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:22.410381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdcf61-e658-4eb0-ae33-81e9d9cc14d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:22.414808 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdcf61-e658-4eb0-ae33-81e9d9cc14d0 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:22.415105 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdcf61-e658-4eb0-ae33-81e9d9cc14d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:22.415419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdcf61-e658-4eb0-ae33-81e9d9cc14d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:22.418218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cdcf61-e658-4eb0-ae33-81e9d9cc14d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:22.419772 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cdcf61-e658-4eb0-ae33-81e9d9cc14d0 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:22.452934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d699c7a9-5c58-432e-b45a-7cbb8bfdbc1a None None] MFA Rules not processed for user `4d7c1adbb2ad456eaf5793a15d6fcb4b`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:22.456719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d699c7a9-5c58-432e-b45a-7cbb8bfdbc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:22.466956 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 857/1735] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:22.467925 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdcf61-e658-4eb0-ae33-81e9d9cc14d0 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:22.505384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdcf61-e658-4eb0-ae33-81e9d9cc14d0 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:22.522175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48fccb-6845-4e56-80b9-8d721e4cb503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:22.522759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48fccb-6845-4e56-80b9-8d721e4cb503 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:22.523049 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48fccb-6845-4e56-80b9-8d721e4cb503 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:22.523506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48fccb-6845-4e56-80b9-8d721e4cb503 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:22.524230 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d48fccb-6845-4e56-80b9-8d721e4cb503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:22.527212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d48fccb-6845-4e56-80b9-8d721e4cb503 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:22.669885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 879/1736] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:22.671781 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699c7a9-5c58-432e-b45a-7cbb8bfdbc1a None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:22.726290 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 858/1737] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:22 2022] POST /identity/v3/projects => generated 385 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:22.729700 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48fccb-6845-4e56-80b9-8d721e4cb503 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:22.741447 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699c7a9-5c58-432e-b45a-7cbb8bfdbc1a None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:22.804655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:22.834433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:22.884847 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e707c908-229a-416e-8aaf-a4185189ee93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:22.885624 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707c908-229a-416e-8aaf-a4185189ee93 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:22.886020 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707c908-229a-416e-8aaf-a4185189ee93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:22.886454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e707c908-229a-416e-8aaf-a4185189ee93 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:22.887686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707c908-229a-416e-8aaf-a4185189ee93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:22.889856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e707c908-229a-416e-8aaf-a4185189ee93 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:22.911740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 880/1738] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:22.913625 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7265d-1e72-445c-8478-e6eecdc6ff43 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:22.914164 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7265d-1e72-445c-8478-e6eecdc6ff43 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:22.914854 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7265d-1e72-445c-8478-e6eecdc6ff43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:22.950783 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f4d7265d-1e72-445c-8478-e6eecdc6ff43 None None] MFA Rules not processed for user `27cc3d38926f470c91eeb6aecfc86a48`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:22.952319 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f4d7265d-1e72-445c-8478-e6eecdc6ff43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:22.976799 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48fccb-6845-4e56-80b9-8d721e4cb503 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:22.992893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999750e-35b9-4f40-8979-03e8e79eedee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:22.993817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999750e-35b9-4f40-8979-03e8e79eedee admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:22.994093 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999750e-35b9-4f40-8979-03e8e79eedee admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:22.994693 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999750e-35b9-4f40-8979-03e8e79eedee admin admin] PATH_INFO: `/v3/projects/c650c8e4d52f4d448aeac0e541ceec6b/users/ff745a46f6ff485eae9f8b424a769e0e/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:23.028426 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999750e-35b9-4f40-8979-03e8e79eedee admin admin] RBAC: Authorizing `identity:create_grant(project_id=c650c8e4d52f4d448aeac0e541ceec6b, user_id=ff745a46f6ff485eae9f8b424a769e0e, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:23.033664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999750e-35b9-4f40-8979-03e8e79eedee admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:23.059593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 859/1739] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:22 2022] PUT /identity/v3/projects/c650c8e4d52f4d448aeac0e541ceec6b/users/ff745a46f6ff485eae9f8b424a769e0e/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:23.060720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999750e-35b9-4f40-8979-03e8e79eedee admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:23.108873 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 881/1740] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:23.108873 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7265d-1e72-445c-8478-e6eecdc6ff43 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:23.148903 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7265d-1e72-445c-8478-e6eecdc6ff43 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:23.215497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999750e-35b9-4f40-8979-03e8e79eedee admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:23.227980 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e400848-76bb-4867-a239-fcc1c80b3e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:23.228917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e400848-76bb-4867-a239-fcc1c80b3e0b admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:23.229826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e400848-76bb-4867-a239-fcc1c80b3e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:23.230106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e400848-76bb-4867-a239-fcc1c80b3e0b admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:23.232037 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e400848-76bb-4867-a239-fcc1c80b3e0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:23.236591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e400848-76bb-4867-a239-fcc1c80b3e0b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:23.289959 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 860/1741] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:23 2022] POST /identity/v3/users => generated 299 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:23.294384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e400848-76bb-4867-a239-fcc1c80b3e0b admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:23.339939 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e400848-76bb-4867-a239-fcc1c80b3e0b admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:23.352167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3955a29-7824-45b5-b7dd-78b07c3dd4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:23.354011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3955a29-7824-45b5-b7dd-78b07c3dd4eb admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:23.354777 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3955a29-7824-45b5-b7dd-78b07c3dd4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:23.354777 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3955a29-7824-45b5-b7dd-78b07c3dd4eb admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:23.357081 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3955a29-7824-45b5-b7dd-78b07c3dd4eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:23.358774 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3955a29-7824-45b5-b7dd-78b07c3dd4eb admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:23.362346 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3333e8-81e7-4cc2-b7eb-4c959066d0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:23.363751 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3333e8-81e7-4cc2-b7eb-4c959066d0cc admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:23.365181 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3333e8-81e7-4cc2-b7eb-4c959066d0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:23.365613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3333e8-81e7-4cc2-b7eb-4c959066d0cc admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:23.378611 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3333e8-81e7-4cc2-b7eb-4c959066d0cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:23.382537 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3333e8-81e7-4cc2-b7eb-4c959066d0cc admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:23.408568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 861/1742] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:23.409521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3955a29-7824-45b5-b7dd-78b07c3dd4eb admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:23.428101 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 882/1743] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:23.430448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3955a29-7824-45b5-b7dd-78b07c3dd4eb admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:23.431544 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3333e8-81e7-4cc2-b7eb-4c959066d0cc admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:23.445432 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485c6e8-40e1-4e99-ba44-6c99d15fec40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:23.446133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485c6e8-40e1-4e99-ba44-6c99d15fec40 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:23.446406 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485c6e8-40e1-4e99-ba44-6c99d15fec40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:23.446688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0485c6e8-40e1-4e99-ba44-6c99d15fec40 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:23.447524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485c6e8-40e1-4e99-ba44-6c99d15fec40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:23.449151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0485c6e8-40e1-4e99-ba44-6c99d15fec40 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:23.481294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3333e8-81e7-4cc2-b7eb-4c959066d0cc admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:23.481975 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 862/1744] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:23.482918 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485c6e8-40e1-4e99-ba44-6c99d15fec40 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:23.502329 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0697f78-79e6-44e6-9690-5f6b694ff294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:23.503652 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0697f78-79e6-44e6-9690-5f6b694ff294 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:23.509689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0485c6e8-40e1-4e99-ba44-6c99d15fec40 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:23.522472 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a45a6a7-d7af-4343-9aa6-63766373bc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:23.523544 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45a6a7-d7af-4343-9aa6-63766373bc42 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:23.523705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45a6a7-d7af-4343-9aa6-63766373bc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:23.524039 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a45a6a7-d7af-4343-9aa6-63766373bc42 admin admin] PATH_INFO: `/v3/projects/d3d6afb669de4bf8a4bec4405dac8199/users/3fc2263237cd4a788f8461929c576892/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:23.532096 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0697f78-79e6-44e6-9690-5f6b694ff294 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:23.532421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0697f78-79e6-44e6-9690-5f6b694ff294 admin admin] PATH_INFO: `/v3/projects/c650c8e4d52f4d448aeac0e541ceec6b/users/ff745a46f6ff485eae9f8b424a769e0e/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:23.571841 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0697f78-79e6-44e6-9690-5f6b694ff294 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c650c8e4d52f4d448aeac0e541ceec6b, user_id=ff745a46f6ff485eae9f8b424a769e0e, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:23.592190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0697f78-79e6-44e6-9690-5f6b694ff294 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:23.600382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a45a6a7-d7af-4343-9aa6-63766373bc42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3d6afb669de4bf8a4bec4405dac8199, user_id=3fc2263237cd4a788f8461929c576892, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:23.603062 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a45a6a7-d7af-4343-9aa6-63766373bc42 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:23.638248 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 883/1745] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:23 2022] PUT /identity/v3/projects/c650c8e4d52f4d448aeac0e541ceec6b/users/ff745a46f6ff485eae9f8b424a769e0e/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:23.644396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0697f78-79e6-44e6-9690-5f6b694ff294 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:23.663382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 863/1746] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:23 2022] PUT /identity/v3/projects/d3d6afb669de4bf8a4bec4405dac8199/users/3fc2263237cd4a788f8461929c576892/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:23.663382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe717735-5218-431f-bc11-06d33e1637a9 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:23.663382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe717735-5218-431f-bc11-06d33e1637a9 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:23.663382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe717735-5218-431f-bc11-06d33e1637a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:23.745519 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-fe717735-5218-431f-bc11-06d33e1637a9 None None] MFA Rules not processed for user `ff745a46f6ff485eae9f8b424a769e0e`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:23.760182 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-fe717735-5218-431f-bc11-06d33e1637a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:23.896589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:23.918248 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:23.970709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0697f78-79e6-44e6-9690-5f6b694ff294 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:24.008516 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 864/1747] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:23 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:24.010001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fea67-b263-44f7-8f8e-e4cb28650601 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:24.020349 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fea67-b263-44f7-8f8e-e4cb28650601 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:24.020688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fea67-b263-44f7-8f8e-e4cb28650601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:24.137824 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-a62fea67-b263-44f7-8f8e-e4cb28650601 None None] MFA Rules not processed for user `3fc2263237cd4a788f8461929c576892`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:24.139095 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a62fea67-b263-44f7-8f8e-e4cb28650601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:24.214378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1eb4ad-063d-497e-a0d2-5ded3233e09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:24.216359 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1eb4ad-063d-497e-a0d2-5ded3233e09c admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:24.216774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1eb4ad-063d-497e-a0d2-5ded3233e09c admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:24.217193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1eb4ad-063d-497e-a0d2-5ded3233e09c admin admin] PATH_INFO: `/v3/projects/51ac2812dffc49d1974b61e28d0d4ce6/users/c1904802b5f648aeb335f0f08fe43a15/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:24.260380 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1eb4ad-063d-497e-a0d2-5ded3233e09c admin admin] RBAC: Authorizing `identity:create_grant(project_id=51ac2812dffc49d1974b61e28d0d4ce6, user_id=c1904802b5f648aeb335f0f08fe43a15, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:24.264464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1eb4ad-063d-497e-a0d2-5ded3233e09c admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:24.306199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 884/1748] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:23 2022] PUT /identity/v3/projects/51ac2812dffc49d1974b61e28d0d4ce6/users/c1904802b5f648aeb335f0f08fe43a15/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:24.313150 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32517a-dac7-4efd-bdcc-59c2fdb64711 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:24.313891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32517a-dac7-4efd-bdcc-59c2fdb64711 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:24.314918 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32517a-dac7-4efd-bdcc-59c2fdb64711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:24.349967 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-0e32517a-dac7-4efd-bdcc-59c2fdb64711 None None] MFA Rules not processed for user `955c44ffe1bd426e900c7405df44d1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:24.352754 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 865/1749] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:24.354593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fea67-b263-44f7-8f8e-e4cb28650601 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:24.358452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a62fea67-b263-44f7-8f8e-e4cb28650601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:24.374936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-0e32517a-dac7-4efd-bdcc-59c2fdb64711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:24.530868 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 885/1750] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jun 22 11:45:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:24.531927 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32517a-dac7-4efd-bdcc-59c2fdb64711 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:24.538868 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fea67-b263-44f7-8f8e-e4cb28650601 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:24.634280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32517a-dac7-4efd-bdcc-59c2fdb64711 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:24.643075 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab7eef-f66e-4918-a341-83ad4c21787c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:24.643990 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab7eef-f66e-4918-a341-83ad4c21787c admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:24.644387 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab7eef-f66e-4918-a341-83ad4c21787c admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:24.645736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab7eef-f66e-4918-a341-83ad4c21787c admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:24.648556 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab7eef-f66e-4918-a341-83ad4c21787c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:24.650548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab7eef-f66e-4918-a341-83ad4c21787c admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:24.678152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 886/1751] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:24.684194 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533cfe7-28d4-4c2e-b4d0-9d62043cbe6e None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:24.684656 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533cfe7-28d4-4c2e-b4d0-9d62043cbe6e None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:24.685082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533cfe7-28d4-4c2e-b4d0-9d62043cbe6e None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:24.691476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 887/1752] 10.222.0.36 () {54 vars in 912 bytes} [Wed Jun 22 11:45:24 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:45:24.693682 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533cfe7-28d4-4c2e-b4d0-9d62043cbe6e None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:24.730725 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f0348-31c3-4c01-8196-4c203dbefeaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:24.732309 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f0348-31c3-4c01-8196-4c203dbefeaa service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:24.733647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f0348-31c3-4c01-8196-4c203dbefeaa service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:24.734630 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533cfe7-28d4-4c2e-b4d0-9d62043cbe6e None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:24.736333 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f0348-31c3-4c01-8196-4c203dbefeaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:24.760664 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c3b7e-9094-4d46-bc32-3f699b1eff0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:24.761623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c3b7e-9094-4d46-bc32-3f699b1eff0d admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:24.762133 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c3b7e-9094-4d46-bc32-3f699b1eff0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:24.762618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c3b7e-9094-4d46-bc32-3f699b1eff0d admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:24.763586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c3b7e-9094-4d46-bc32-3f699b1eff0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:24.765942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c3b7e-9094-4d46-bc32-3f699b1eff0d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:24.796227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:24.824273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 888/1753] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:24 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:24.824273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c3b7e-9094-4d46-bc32-3f699b1eff0d admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:24.901450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:25.009045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f0348-31c3-4c01-8196-4c203dbefeaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:25.016178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f0348-31c3-4c01-8196-4c203dbefeaa service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:25.032118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c3b7e-9094-4d46-bc32-3f699b1eff0d admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:25.039077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb19e94-514c-43a6-8578-53b56bbe85ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:25.040426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb19e94-514c-43a6-8578-53b56bbe85ec admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:25.040849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb19e94-514c-43a6-8578-53b56bbe85ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:25.041314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb19e94-514c-43a6-8578-53b56bbe85ec admin admin] PATH_INFO: `/v3/projects/51ac2812dffc49d1974b61e28d0d4ce6/users/c1904802b5f648aeb335f0f08fe43a15/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:25.044716 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb19e94-514c-43a6-8578-53b56bbe85ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=51ac2812dffc49d1974b61e28d0d4ce6, user_id=c1904802b5f648aeb335f0f08fe43a15, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:25.048220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb19e94-514c-43a6-8578-53b56bbe85ec admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:25.059688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 866/1754] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:24 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:25.067443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 889/1755] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:24 2022] PUT /identity/v3/projects/51ac2812dffc49d1974b61e28d0d4ce6/users/c1904802b5f648aeb335f0f08fe43a15/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:25.068623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb19e94-514c-43a6-8578-53b56bbe85ec admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:25.070634 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-dbb19e94-514c-43a6-8578-53b56bbe85ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:25.078028 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba94f76-494a-478d-9254-904b2d0b0fbc None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:25.078395 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba94f76-494a-478d-9254-904b2d0b0fbc None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:25.078685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba94f76-494a-478d-9254-904b2d0b0fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:25.163789 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-1ba94f76-494a-478d-9254-904b2d0b0fbc None None] MFA Rules not processed for user `c00d0bb777a044a3a4d4540d187d938a`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:25.165028 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-1ba94f76-494a-478d-9254-904b2d0b0fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:25.376827 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 867/1756] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jun 22 11:45:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:25.377497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba94f76-494a-478d-9254-904b2d0b0fbc None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:25.414193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb19e94-514c-43a6-8578-53b56bbe85ec admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:25.453268 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df012a4-3eb0-4573-917b-1eaa09eb5ad2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd642e290e44e67b56eb3b8361d7af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:25.457353 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df012a4-3eb0-4573-917b-1eaa09eb5ad2 service swift] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:25.463550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df012a4-3eb0-4573-917b-1eaa09eb5ad2 service swift] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:25.464101 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df012a4-3eb0-4573-917b-1eaa09eb5ad2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:25.524604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba94f76-494a-478d-9254-904b2d0b0fbc None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:25.553848 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c07ae-b374-40ed-8d64-037c41c4f095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:25.554130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c07ae-b374-40ed-8d64-037c41c4f095 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:25.554417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c07ae-b374-40ed-8d64-037c41c4f095 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:25.554692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c07ae-b374-40ed-8d64-037c41c4f095 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:25.560246 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c07ae-b374-40ed-8d64-037c41c4f095 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:25.571337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c07ae-b374-40ed-8d64-037c41c4f095 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:25.646407 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 868/1757] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:25 2022] POST /identity/v3/users => generated 299 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:25.654570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ae8a5-8498-43b6-9dd0-bc778a7b714f None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:25.654923 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ae8a5-8498-43b6-9dd0-bc778a7b714f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:25.655146 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ae8a5-8498-43b6-9dd0-bc778a7b714f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:25.669398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df012a4-3eb0-4573-917b-1eaa09eb5ad2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:25.671974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df012a4-3eb0-4573-917b-1eaa09eb5ad2 service swift] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:25.686271 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 890/1758] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jun 22 11:45:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 22 11:45:25.687993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df012a4-3eb0-4573-917b-1eaa09eb5ad2 service swift] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:25.689298 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-7df012a4-3eb0-4573-917b-1eaa09eb5ad2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:25.705089 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df012a4-3eb0-4573-917b-1eaa09eb5ad2 service swift] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:25.710368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a591cbc-e770-4f30-9ad7-b4d1b8e4bdce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:25.711635 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a591cbc-e770-4f30-9ad7-b4d1b8e4bdce service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:25.711635 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a591cbc-e770-4f30-9ad7-b4d1b8e4bdce service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:25.712551 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a591cbc-e770-4f30-9ad7-b4d1b8e4bdce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:25.727620 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-214ae8a5-8498-43b6-9dd0-bc778a7b714f None None] MFA Rules not processed for user `c1904802b5f648aeb335f0f08fe43a15`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:25.731028 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-214ae8a5-8498-43b6-9dd0-bc778a7b714f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:25.734085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a591cbc-e770-4f30-9ad7-b4d1b8e4bdce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:25.737245 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a591cbc-e770-4f30-9ad7-b4d1b8e4bdce service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:25.755835 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 891/1759] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:25.757008 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a591cbc-e770-4f30-9ad7-b4d1b8e4bdce service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:25.786003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a591cbc-e770-4f30-9ad7-b4d1b8e4bdce service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:25.796684 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f54941-ebc4-4d59-8daf-ea9f25796894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:25.797374 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f54941-ebc4-4d59-8daf-ea9f25796894 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:25.797683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f54941-ebc4-4d59-8daf-ea9f25796894 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:25.798572 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f54941-ebc4-4d59-8daf-ea9f25796894 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:25.799320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f54941-ebc4-4d59-8daf-ea9f25796894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:25.803569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f54941-ebc4-4d59-8daf-ea9f25796894 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:25.828793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 892/1760] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:25.833009 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f54941-ebc4-4d59-8daf-ea9f25796894 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:25.874465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f54941-ebc4-4d59-8daf-ea9f25796894 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:25.894960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabd5a7-0c43-4f87-ac1d-3d911a5ef40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:25.895885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabd5a7-0c43-4f87-ac1d-3d911a5ef40d admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:25.896172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabd5a7-0c43-4f87-ac1d-3d911a5ef40d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:25.896447 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabd5a7-0c43-4f87-ac1d-3d911a5ef40d admin admin] PATH_INFO: `/v3/projects/dccac589675d408c8fa056b4c088358b/users/9bf0226d2b914535817961f78191c8c4/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:25.908873 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:25.923324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:25.934160 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 869/1761] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:25.952773 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ae8a5-8498-43b6-9dd0-bc778a7b714f None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:25.954685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-214ae8a5-8498-43b6-9dd0-bc778a7b714f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:25.964375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcabd5a7-0c43-4f87-ac1d-3d911a5ef40d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dccac589675d408c8fa056b4c088358b, user_id=9bf0226d2b914535817961f78191c8c4, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:25.967107 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcabd5a7-0c43-4f87-ac1d-3d911a5ef40d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:26.048361 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 893/1762] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:25 2022] PUT /identity/v3/projects/dccac589675d408c8fa056b4c088358b/users/9bf0226d2b914535817961f78191c8c4/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:26.056190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabd5a7-0c43-4f87-ac1d-3d911a5ef40d admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:26.219558 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ae8a5-8498-43b6-9dd0-bc778a7b714f None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:26.272670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabd5a7-0c43-4f87-ac1d-3d911a5ef40d admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:26.289035 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef67fe-1797-4727-aa13-7f64ddafbbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:26.289843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef67fe-1797-4727-aa13-7f64ddafbbcc admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:26.290239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef67fe-1797-4727-aa13-7f64ddafbbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:26.290627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef67fe-1797-4727-aa13-7f64ddafbbcc admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:26.291606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef67fe-1797-4727-aa13-7f64ddafbbcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:26.293420 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef67fe-1797-4727-aa13-7f64ddafbbcc admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:26.340884 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 894/1763] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:26.341962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef67fe-1797-4727-aa13-7f64ddafbbcc admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:26.397501 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef67fe-1797-4727-aa13-7f64ddafbbcc admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:26.410629 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb24f071-e65d-46ad-b6c6-3b26a33c6a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:26.411753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24f071-e65d-46ad-b6c6-3b26a33c6a7a admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:26.412148 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24f071-e65d-46ad-b6c6-3b26a33c6a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:26.413916 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb24f071-e65d-46ad-b6c6-3b26a33c6a7a admin admin] PATH_INFO: `/v3/projects/dccac589675d408c8fa056b4c088358b/users/9bf0226d2b914535817961f78191c8c4/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:26.419719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24f071-e65d-46ad-b6c6-3b26a33c6a7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dccac589675d408c8fa056b4c088358b, user_id=9bf0226d2b914535817961f78191c8c4, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:26.431470 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb24f071-e65d-46ad-b6c6-3b26a33c6a7a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:26.459056 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63528e5d-e52f-4cd6-88d0-10476ddbb8ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:26.459543 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63528e5d-e52f-4cd6-88d0-10476ddbb8ac service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:26.459834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63528e5d-e52f-4cd6-88d0-10476ddbb8ac service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:26.460261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63528e5d-e52f-4cd6-88d0-10476ddbb8ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:26.492454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 895/1764] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:26 2022] PUT /identity/v3/projects/dccac589675d408c8fa056b4c088358b/users/9bf0226d2b914535817961f78191c8c4/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:26.494153 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddabdf-ef7a-45d0-aaf0-1021f6988f9e None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:26.494455 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddabdf-ef7a-45d0-aaf0-1021f6988f9e None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:26.494740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddabdf-ef7a-45d0-aaf0-1021f6988f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:26.540864 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-78ddabdf-ef7a-45d0-aaf0-1021f6988f9e None None] MFA Rules not processed for user `9bf0226d2b914535817961f78191c8c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:26.544776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-78ddabdf-ef7a-45d0-aaf0-1021f6988f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:26.661031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63528e5d-e52f-4cd6-88d0-10476ddbb8ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:26.662498 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63528e5d-e52f-4cd6-88d0-10476ddbb8ac service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:26.682702 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 870/1765] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 731 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:45:26.718810 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 896/1766] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:26.736314 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63528e5d-e52f-4cd6-88d0-10476ddbb8ac service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:26.751692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63528e5d-e52f-4cd6-88d0-10476ddbb8ac service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:26.853493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44916ff0-5cba-4d18-bf46-699311751771 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:26.854223 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44916ff0-5cba-4d18-bf46-699311751771 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:26.854514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44916ff0-5cba-4d18-bf46-699311751771 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:26.854795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44916ff0-5cba-4d18-bf46-699311751771 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:26.866311 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddabdf-ef7a-45d0-aaf0-1021f6988f9e None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:26.914442 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44916ff0-5cba-4d18-bf46-699311751771 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:26.916659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44916ff0-5cba-4d18-bf46-699311751771 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:26.976864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 871/1767] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:27.006730 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:27.033370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddabdf-ef7a-45d0-aaf0-1021f6988f9e None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:27.047704 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041e3a6-864d-4950-8973-b2c78317a5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:27.051870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041e3a6-864d-4950-8973-b2c78317a5c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:27.052449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041e3a6-864d-4950-8973-b2c78317a5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:27.052449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041e3a6-864d-4950-8973-b2c78317a5c2 admin admin] PATH_INFO: `/v3/users/d0c7435796d74c6781e4903cf2aedc14` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:27.056306 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041e3a6-864d-4950-8973-b2c78317a5c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0c7435796d74c6781e4903cf2aedc14)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:27.060005 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f041e3a6-864d-4950-8973-b2c78317a5c2 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:27.218568 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f041e3a6-864d-4950-8973-b2c78317a5c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0c7435796d74c6781e4903cf2aedc14'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:27.286399 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f041e3a6-864d-4950-8973-b2c78317a5c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0c7435796d74c6781e4903cf2aedc14'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:27.303758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f041e3a6-864d-4950-8973-b2c78317a5c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0c7435796d74c6781e4903cf2aedc14'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:27.340695 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f041e3a6-864d-4950-8973-b2c78317a5c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0c7435796d74c6781e4903cf2aedc14'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:27.346457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f041e3a6-864d-4950-8973-b2c78317a5c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0c7435796d74c6781e4903cf2aedc14'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:27.380910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 897/1768] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:26 2022] DELETE /identity/v3/users/d0c7435796d74c6781e4903cf2aedc14 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:27.384826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b935a-a938-4a78-a217-3971e87b16af None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:27.385211 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b935a-a938-4a78-a217-3971e87b16af None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:27.385442 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b935a-a938-4a78-a217-3971e87b16af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:27.454405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-063b935a-a938-4a78-a217-3971e87b16af None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:27.455800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-063b935a-a938-4a78-a217-3971e87b16af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:27.476779 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:27.628931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 872/1769] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:27.644491 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041e3a6-864d-4950-8973-b2c78317a5c2 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:27.646723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f041e3a6-864d-4950-8973-b2c78317a5c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:27.836910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041e3a6-864d-4950-8973-b2c78317a5c2 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:27.854751 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683ba3d-25cc-45c6-a873-33442a3ec3b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:27.855561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683ba3d-25cc-45c6-a873-33442a3ec3b6 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:27.855863 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683ba3d-25cc-45c6-a873-33442a3ec3b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:27.856146 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683ba3d-25cc-45c6-a873-33442a3ec3b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:27.883565 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c683ba3d-25cc-45c6-a873-33442a3ec3b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:27.885166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c683ba3d-25cc-45c6-a873-33442a3ec3b6 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:27.919031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 898/1770] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:28.500449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b935a-a938-4a78-a217-3971e87b16af None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:28.513352 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-063b935a-a938-4a78-a217-3971e87b16af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:28.547491 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683ba3d-25cc-45c6-a873-33442a3ec3b6 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:28.549500 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-c683ba3d-25cc-45c6-a873-33442a3ec3b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:28.578591 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683ba3d-25cc-45c6-a873-33442a3ec3b6 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:28.591330 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bc220-fd36-4e15-88e6-aebd781d5daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:28.592086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bc220-fd36-4e15-88e6-aebd781d5daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:28.592491 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bc220-fd36-4e15-88e6-aebd781d5daa admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:28.594614 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bc220-fd36-4e15-88e6-aebd781d5daa admin admin] PATH_INFO: `/v3/projects/5e577782a02c4e55ab90699dc6401bce` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:28.599206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02bc220-fd36-4e15-88e6-aebd781d5daa admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e577782a02c4e55ab90699dc6401bce)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:28.601797 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02bc220-fd36-4e15-88e6-aebd781d5daa admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:28.655946 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b935a-a938-4a78-a217-3971e87b16af None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:28.665650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270fc0b-87b7-4b99-9490-35b92f0494f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:28.666331 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270fc0b-87b7-4b99-9490-35b92f0494f1 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:28.666613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270fc0b-87b7-4b99-9490-35b92f0494f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:28.666892 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270fc0b-87b7-4b99-9490-35b92f0494f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:28.669713 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-3270fc0b-87b7-4b99-9490-35b92f0494f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:28.681786 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f02bc220-fd36-4e15-88e6-aebd781d5daa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e577782a02c4e55ab90699dc6401bce', 'request_id': 'req-f02bc220-fd36-4e15-88e6-aebd781d5daa'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:28.693895 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f02bc220-fd36-4e15-88e6-aebd781d5daa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e577782a02c4e55ab90699dc6401bce', 'request_id': 'req-f02bc220-fd36-4e15-88e6-aebd781d5daa'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:28.705832 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f02bc220-fd36-4e15-88e6-aebd781d5daa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e577782a02c4e55ab90699dc6401bce', 'request_id': 'req-f02bc220-fd36-4e15-88e6-aebd781d5daa'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:28.712112 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:28.725246 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f02bc220-fd36-4e15-88e6-aebd781d5daa admin admin] The token cache is being invalidate because project 5e577782a02c4e55ab90699dc6401bce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:28.725582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f02bc220-fd36-4e15-88e6-aebd781d5daa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:28.728402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f02bc220-fd36-4e15-88e6-aebd781d5daa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:28.730627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 899/1771] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:28 2022] DELETE /identity/v3/projects/5e577782a02c4e55ab90699dc6401bce => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:28.735929 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bc220-fd36-4e15-88e6-aebd781d5daa admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:28.738194 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f02bc220-fd36-4e15-88e6-aebd781d5daa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:28.837883 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270fc0b-87b7-4b99-9490-35b92f0494f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:28.839396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270fc0b-87b7-4b99-9490-35b92f0494f1 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:28.865658 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 873/1772] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:28.977486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bc220-fd36-4e15-88e6-aebd781d5daa admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:28.997354 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f64a9b-ee52-4c74-89a6-41db48469197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:28.998533 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f64a9b-ee52-4c74-89a6-41db48469197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:28.998967 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f64a9b-ee52-4c74-89a6-41db48469197 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:28.999422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f64a9b-ee52-4c74-89a6-41db48469197 admin admin] PATH_INFO: `/v3/users/c1904802b5f648aeb335f0f08fe43a15` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:29.001414 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f64a9b-ee52-4c74-89a6-41db48469197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1904802b5f648aeb335f0f08fe43a15)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:29.003575 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f64a9b-ee52-4c74-89a6-41db48469197 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:29.138066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-40f64a9b-ee52-4c74-89a6-41db48469197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1904802b5f648aeb335f0f08fe43a15'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:29.163625 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270fc0b-87b7-4b99-9490-35b92f0494f1 service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:29.166038 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-3270fc0b-87b7-4b99-9490-35b92f0494f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:29.197356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270fc0b-87b7-4b99-9490-35b92f0494f1 service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:29.199229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-40f64a9b-ee52-4c74-89a6-41db48469197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1904802b5f648aeb335f0f08fe43a15'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:29.210818 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db958d9-4714-49c9-8013-4b62d9ebe321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:29.213398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db958d9-4714-49c9-8013-4b62d9ebe321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:29.213699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db958d9-4714-49c9-8013-4b62d9ebe321 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:29.214935 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db958d9-4714-49c9-8013-4b62d9ebe321 admin admin] PATH_INFO: `/v3/users/5e74ad4e32e74ebc91bf4daa1bc0dce2` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:29.218612 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db958d9-4714-49c9-8013-4b62d9ebe321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e74ad4e32e74ebc91bf4daa1bc0dce2)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:29.225401 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db958d9-4714-49c9-8013-4b62d9ebe321 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:29.228136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-40f64a9b-ee52-4c74-89a6-41db48469197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1904802b5f648aeb335f0f08fe43a15'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:29.241217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-40f64a9b-ee52-4c74-89a6-41db48469197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1904802b5f648aeb335f0f08fe43a15'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:29.242281 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-40f64a9b-ee52-4c74-89a6-41db48469197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1904802b5f648aeb335f0f08fe43a15'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:29.264704 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 900/1773] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:28 2022] DELETE /identity/v3/users/c1904802b5f648aeb335f0f08fe43a15 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:29.350291 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9db958d9-4714-49c9-8013-4b62d9ebe321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e74ad4e32e74ebc91bf4daa1bc0dce2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:29.413613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:29.431442 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9db958d9-4714-49c9-8013-4b62d9ebe321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e74ad4e32e74ebc91bf4daa1bc0dce2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:29.443242 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9db958d9-4714-49c9-8013-4b62d9ebe321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e74ad4e32e74ebc91bf4daa1bc0dce2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:29.444102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9db958d9-4714-49c9-8013-4b62d9ebe321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e74ad4e32e74ebc91bf4daa1bc0dce2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:29.463859 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9db958d9-4714-49c9-8013-4b62d9ebe321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e74ad4e32e74ebc91bf4daa1bc0dce2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:29.514997 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c2539-cad8-4ee5-a3fe-38305a9154a1 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:29.515192 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 874/1774] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:29 2022] DELETE /identity/v3/users/5e74ad4e32e74ebc91bf4daa1bc0dce2 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:29.516441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c2539-cad8-4ee5-a3fe-38305a9154a1 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:29.516953 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c2539-cad8-4ee5-a3fe-38305a9154a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:29.552017 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-028c2539-cad8-4ee5-a3fe-38305a9154a1 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:29.559276 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-028c2539-cad8-4ee5-a3fe-38305a9154a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:29.677047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61871596-ea3b-44f4-955b-d22b75d8e7b9 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:29.677667 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61871596-ea3b-44f4-955b-d22b75d8e7b9 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:29.678115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61871596-ea3b-44f4-955b-d22b75d8e7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:29.705054 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 901/1775] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:29.747373 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-61871596-ea3b-44f4-955b-d22b75d8e7b9 None None] MFA Rules not processed for user `c00d0bb777a044a3a4d4540d187d938a`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:29.751983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-61871596-ea3b-44f4-955b-d22b75d8e7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:29.811722 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c2539-cad8-4ee5-a3fe-38305a9154a1 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:29.813871 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-028c2539-cad8-4ee5-a3fe-38305a9154a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:29.868131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:29.980734 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 875/1776] 10.222.0.62 () {58 vars in 1015 bytes} [Wed Jun 22 11:45:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:45:30.023894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c2539-cad8-4ee5-a3fe-38305a9154a1 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:30.052825 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7298d0-154f-4ef6-a9a3-bef784346c01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:30.053510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7298d0-154f-4ef6-a9a3-bef784346c01 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:30.053976 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7298d0-154f-4ef6-a9a3-bef784346c01 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:30.054265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7298d0-154f-4ef6-a9a3-bef784346c01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:30.080729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7298d0-154f-4ef6-a9a3-bef784346c01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:30.082252 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7298d0-154f-4ef6-a9a3-bef784346c01 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:30.118728 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 902/1777] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:45:30.199053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61871596-ea3b-44f4-955b-d22b75d8e7b9 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:30.208835 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61871596-ea3b-44f4-955b-d22b75d8e7b9 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:30.224093 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7723d-0740-45a7-967f-828a4210180e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:30.224787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7723d-0740-45a7-967f-828a4210180e service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:30.225072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7723d-0740-45a7-967f-828a4210180e service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:30.225353 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7723d-0740-45a7-967f-828a4210180e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:30.252614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7723d-0740-45a7-967f-828a4210180e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:30.254305 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7723d-0740-45a7-967f-828a4210180e service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:30.269088 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 876/1778] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:31.327740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7298d0-154f-4ef6-a9a3-bef784346c01 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:31.331105 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5a7298d0-154f-4ef6-a9a3-bef784346c01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:31.362615 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7298d0-154f-4ef6-a9a3-bef784346c01 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:31.390681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26dc6ba-2ef6-4d64-bf4c-9adde9e0b4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:31.392390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26dc6ba-2ef6-4d64-bf4c-9adde9e0b4ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:31.392678 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26dc6ba-2ef6-4d64-bf4c-9adde9e0b4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:31.392956 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26dc6ba-2ef6-4d64-bf4c-9adde9e0b4ff admin admin] PATH_INFO: `/v3/projects/51ac2812dffc49d1974b61e28d0d4ce6` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:31.396632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26dc6ba-2ef6-4d64-bf4c-9adde9e0b4ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=51ac2812dffc49d1974b61e28d0d4ce6)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:31.398586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26dc6ba-2ef6-4d64-bf4c-9adde9e0b4ff admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:31.533172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-a26dc6ba-2ef6-4d64-bf4c-9adde9e0b4ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51ac2812dffc49d1974b61e28d0d4ce6', 'request_id': 'req-a26dc6ba-2ef6-4d64-bf4c-9adde9e0b4ff'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:31.549123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-a26dc6ba-2ef6-4d64-bf4c-9adde9e0b4ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51ac2812dffc49d1974b61e28d0d4ce6', 'request_id': 'req-a26dc6ba-2ef6-4d64-bf4c-9adde9e0b4ff'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:31.568142 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-a26dc6ba-2ef6-4d64-bf4c-9adde9e0b4ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51ac2812dffc49d1974b61e28d0d4ce6', 'request_id': 'req-a26dc6ba-2ef6-4d64-bf4c-9adde9e0b4ff'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:31.599461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-a26dc6ba-2ef6-4d64-bf4c-9adde9e0b4ff admin admin] The token cache is being invalidate because project 51ac2812dffc49d1974b61e28d0d4ce6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:31.599461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-a26dc6ba-2ef6-4d64-bf4c-9adde9e0b4ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:31.599461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-a26dc6ba-2ef6-4d64-bf4c-9adde9e0b4ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:31.599461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 903/1779] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:31 2022] DELETE /identity/v3/projects/51ac2812dffc49d1974b61e28d0d4ce6 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:31.604024 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3ee9b-07dc-4857-b376-f33bef0e5b21 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:31.604509 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3ee9b-07dc-4857-b376-f33bef0e5b21 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:31.604590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3ee9b-07dc-4857-b376-f33bef0e5b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:31.649521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-baf3ee9b-07dc-4857-b376-f33bef0e5b21 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:31.650897 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-baf3ee9b-07dc-4857-b376-f33bef0e5b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:31.826875 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 877/1780] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:31.835071 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201aa92-190d-42b5-88b8-d802f68b426f None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:31.835664 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201aa92-190d-42b5-88b8-d802f68b426f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:31.836254 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201aa92-190d-42b5-88b8-d802f68b426f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:31.889422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-a201aa92-190d-42b5-88b8-d802f68b426f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:31.893314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-a201aa92-190d-42b5-88b8-d802f68b426f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:31.918737 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3ee9b-07dc-4857-b376-f33bef0e5b21 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:31.925514 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 904/1781] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:31.956760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3ee9b-07dc-4857-b376-f33bef0e5b21 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:31.976037 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c835f-f345-4c98-9ac2-0f90579f5970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:31.977068 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c835f-f345-4c98-9ac2-0f90579f5970 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:31.977560 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c835f-f345-4c98-9ac2-0f90579f5970 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:31.977980 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c835f-f345-4c98-9ac2-0f90579f5970 admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:31.978866 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c835f-f345-4c98-9ac2-0f90579f5970 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:31.980005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c835f-f345-4c98-9ac2-0f90579f5970 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:31.982249 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c835f-f345-4c98-9ac2-0f90579f5970 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:32.015389 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 878/1782] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:45:32.020345 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201aa92-190d-42b5-88b8-d802f68b426f None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:32.048346 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201aa92-190d-42b5-88b8-d802f68b426f None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:32.073333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627f811-73d3-407f-b0bd-c40c855812ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:32.074651 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627f811-73d3-407f-b0bd-c40c855812ff admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:32.075751 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627f811-73d3-407f-b0bd-c40c855812ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:32.077388 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627f811-73d3-407f-b0bd-c40c855812ff admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:32.081197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627f811-73d3-407f-b0bd-c40c855812ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:32.085246 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627f811-73d3-407f-b0bd-c40c855812ff admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:32.170607 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 905/1783] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:32 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:32.172084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c835f-f345-4c98-9ac2-0f90579f5970 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:32.195050 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c835f-f345-4c98-9ac2-0f90579f5970 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:32.445686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59743970-d650-49f5-9f87-432b1d5cc574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:32.446502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59743970-d650-49f5-9f87-432b1d5cc574 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:32.446919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59743970-d650-49f5-9f87-432b1d5cc574 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:32.447207 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59743970-d650-49f5-9f87-432b1d5cc574 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:32.447972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59743970-d650-49f5-9f87-432b1d5cc574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:32.451129 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59743970-d650-49f5-9f87-432b1d5cc574 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:32.460124 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:32.486922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627f811-73d3-407f-b0bd-c40c855812ff admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:32.489549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 879/1784] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:32 2022] POST /identity/v3/users => generated 294 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:32.507761 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627f811-73d3-407f-b0bd-c40c855812ff admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:32.517774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ad2b1-0cdf-48d5-8d96-176ae095bcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:32.518478 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ad2b1-0cdf-48d5-8d96-176ae095bcab admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:32.518760 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ad2b1-0cdf-48d5-8d96-176ae095bcab admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:32.519210 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ad2b1-0cdf-48d5-8d96-176ae095bcab admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:32.520261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ad2b1-0cdf-48d5-8d96-176ae095bcab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:32.522872 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ad2b1-0cdf-48d5-8d96-176ae095bcab admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:32.575013 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 906/1785] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:32.579314 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59743970-d650-49f5-9f87-432b1d5cc574 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:32.598261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59743970-d650-49f5-9f87-432b1d5cc574 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:32.611480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c537dba-b617-475c-a275-845eec0c1ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:32.612945 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c537dba-b617-475c-a275-845eec0c1ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:32.613390 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c537dba-b617-475c-a275-845eec0c1ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:32.613830 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c537dba-b617-475c-a275-845eec0c1ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:32.614723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c537dba-b617-475c-a275-845eec0c1ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:32.624285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c537dba-b617-475c-a275-845eec0c1ed7 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:32.645517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 880/1786] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:45:32.650990 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ad2b1-0cdf-48d5-8d96-176ae095bcab admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:32.688126 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ad2b1-0cdf-48d5-8d96-176ae095bcab admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:32.701991 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26429ab-7c7d-4054-bdb0-4d2e0af4a182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:32.702962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26429ab-7c7d-4054-bdb0-4d2e0af4a182 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:32.703254 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26429ab-7c7d-4054-bdb0-4d2e0af4a182 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:32.704333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26429ab-7c7d-4054-bdb0-4d2e0af4a182 admin admin] PATH_INFO: `/v3/projects/9d344db5c7a9431cad3e975c128ce7d9/users/4087a6b4a4024774b3f0961406ee5222/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:32.763543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26429ab-7c7d-4054-bdb0-4d2e0af4a182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d344db5c7a9431cad3e975c128ce7d9, user_id=4087a6b4a4024774b3f0961406ee5222, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:32.767667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26429ab-7c7d-4054-bdb0-4d2e0af4a182 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:32.801971 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 907/1787] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:32 2022] PUT /identity/v3/projects/9d344db5c7a9431cad3e975c128ce7d9/users/4087a6b4a4024774b3f0961406ee5222/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:32.815083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc198123-5fd3-4543-a5d0-3b2bc2e08aec None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:32.815303 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc198123-5fd3-4543-a5d0-3b2bc2e08aec None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:32.815663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc198123-5fd3-4543-a5d0-3b2bc2e08aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:32.859526 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-fc198123-5fd3-4543-a5d0-3b2bc2e08aec None None] MFA Rules not processed for user `4087a6b4a4024774b3f0961406ee5222`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:32.862844 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-fc198123-5fd3-4543-a5d0-3b2bc2e08aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:33.160679 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 881/1788] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:33.162869 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26429ab-7c7d-4054-bdb0-4d2e0af4a182 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:33.330381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc198123-5fd3-4543-a5d0-3b2bc2e08aec None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:33.333399 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-fc198123-5fd3-4543-a5d0-3b2bc2e08aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:33.369521 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:33.476760 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26429ab-7c7d-4054-bdb0-4d2e0af4a182 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:33.501674 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c36a6c-9385-4116-90b6-50698f30d325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:33.507313 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c36a6c-9385-4116-90b6-50698f30d325 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:33.516148 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c36a6c-9385-4116-90b6-50698f30d325 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:33.521219 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c36a6c-9385-4116-90b6-50698f30d325 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:33.523038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c36a6c-9385-4116-90b6-50698f30d325 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:33.534148 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c36a6c-9385-4116-90b6-50698f30d325 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:33.567839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc198123-5fd3-4543-a5d0-3b2bc2e08aec None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:33.589228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f0ac6-c453-4e8a-930c-a0cb3cf93b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:33.589922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f0ac6-c453-4e8a-930c-a0cb3cf93b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:33.593717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f0ac6-c453-4e8a-930c-a0cb3cf93b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:33.594026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f0ac6-c453-4e8a-930c-a0cb3cf93b0c admin admin] PATH_INFO: `/v3/projects/a37fd61419f7409689eb7bc3807bf16c` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:33.599972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f0ac6-c453-4e8a-930c-a0cb3cf93b0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a37fd61419f7409689eb7bc3807bf16c)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:33.606929 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f0ac6-c453-4e8a-930c-a0cb3cf93b0c admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:33.654571 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 908/1789] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:33 2022] POST /identity/v3/projects => generated 385 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:33.658540 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c36a6c-9385-4116-90b6-50698f30d325 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:33.696603 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cf4f0ac6-c453-4e8a-930c-a0cb3cf93b0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a37fd61419f7409689eb7bc3807bf16c', 'request_id': 'req-cf4f0ac6-c453-4e8a-930c-a0cb3cf93b0c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:33.710051 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cf4f0ac6-c453-4e8a-930c-a0cb3cf93b0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a37fd61419f7409689eb7bc3807bf16c', 'request_id': 'req-cf4f0ac6-c453-4e8a-930c-a0cb3cf93b0c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:33.723686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cf4f0ac6-c453-4e8a-930c-a0cb3cf93b0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a37fd61419f7409689eb7bc3807bf16c', 'request_id': 'req-cf4f0ac6-c453-4e8a-930c-a0cb3cf93b0c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:33.736436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:33.753623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cf4f0ac6-c453-4e8a-930c-a0cb3cf93b0c admin admin] The token cache is being invalidate because project a37fd61419f7409689eb7bc3807bf16c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:33.756363 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cf4f0ac6-c453-4e8a-930c-a0cb3cf93b0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:33.758896 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cf4f0ac6-c453-4e8a-930c-a0cb3cf93b0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:33.768363 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 882/1790] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:33 2022] DELETE /identity/v3/projects/a37fd61419f7409689eb7bc3807bf16c => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:33.773728 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f0ac6-c453-4e8a-930c-a0cb3cf93b0c admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:33.793562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-cf4f0ac6-c453-4e8a-930c-a0cb3cf93b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:33.950321 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c36a6c-9385-4116-90b6-50698f30d325 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:33.993749 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f0ac6-c453-4e8a-930c-a0cb3cf93b0c admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:34.028754 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040fe0ba-04f4-437e-89dd-1b18a50c9dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:34.032473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040fe0ba-04f4-437e-89dd-1b18a50c9dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:34.033422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040fe0ba-04f4-437e-89dd-1b18a50c9dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:34.034314 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040fe0ba-04f4-437e-89dd-1b18a50c9dc4 admin admin] PATH_INFO: `/v3/users/ff745a46f6ff485eae9f8b424a769e0e` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:34.043971 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040fe0ba-04f4-437e-89dd-1b18a50c9dc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff745a46f6ff485eae9f8b424a769e0e)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:34.051083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040fe0ba-04f4-437e-89dd-1b18a50c9dc4 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:34.212512 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c11bbc-8082-4d80-bba4-cfb22a737ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:34.213371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c11bbc-8082-4d80-bba4-cfb22a737ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:34.213514 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c11bbc-8082-4d80-bba4-cfb22a737ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:34.213830 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c11bbc-8082-4d80-bba4-cfb22a737ab3 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:34.214595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c11bbc-8082-4d80-bba4-cfb22a737ab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:34.216815 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c11bbc-8082-4d80-bba4-cfb22a737ab3 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:34.267781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-040fe0ba-04f4-437e-89dd-1b18a50c9dc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff745a46f6ff485eae9f8b424a769e0e'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:34.305696 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-040fe0ba-04f4-437e-89dd-1b18a50c9dc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff745a46f6ff485eae9f8b424a769e0e'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:34.326847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-040fe0ba-04f4-437e-89dd-1b18a50c9dc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff745a46f6ff485eae9f8b424a769e0e'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:34.328370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-040fe0ba-04f4-437e-89dd-1b18a50c9dc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff745a46f6ff485eae9f8b424a769e0e'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:34.352393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-040fe0ba-04f4-437e-89dd-1b18a50c9dc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff745a46f6ff485eae9f8b424a769e0e'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:34.357767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 909/1791] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:33 2022] POST /identity/v3/users => generated 299 bytes in 700 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:34.373134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c11bbc-8082-4d80-bba4-cfb22a737ab3 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:34.386841 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 883/1792] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:33 2022] DELETE /identity/v3/users/ff745a46f6ff485eae9f8b424a769e0e => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:34.402186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-c8c11bbc-8082-4d80-bba4-cfb22a737ab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:34.438608 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78516b54-9448-4f1d-b497-74f6c1aa30cc None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:34.439128 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78516b54-9448-4f1d-b497-74f6c1aa30cc None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:34.439904 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78516b54-9448-4f1d-b497-74f6c1aa30cc None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:34.442229 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 884/1793] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jun 22 11:45:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:45:34.444624 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c11bbc-8082-4d80-bba4-cfb22a737ab3 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:34.524304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40592b9b-819d-4360-af2c-438f92fd39b4 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:34.527570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40592b9b-819d-4360-af2c-438f92fd39b4 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:34.527570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40592b9b-819d-4360-af2c-438f92fd39b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:34.565507 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008d229-b004-459a-8648-8ae2797246d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:34.566970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008d229-b004-459a-8648-8ae2797246d5 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:34.567454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008d229-b004-459a-8648-8ae2797246d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:34.567887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008d229-b004-459a-8648-8ae2797246d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:34.568784 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008d229-b004-459a-8648-8ae2797246d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:34.570564 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008d229-b004-459a-8648-8ae2797246d5 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:34.572619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:34.597017 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 910/1794] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:34.598171 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008d229-b004-459a-8648-8ae2797246d5 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:34.600661 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-40592b9b-819d-4360-af2c-438f92fd39b4 None None] MFA Rules not processed for user `479a376dcac14c509c1c55f0046d88cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:34.603393 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d008d229-b004-459a-8648-8ae2797246d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:34.605345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-40592b9b-819d-4360-af2c-438f92fd39b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:34.626777 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008d229-b004-459a-8648-8ae2797246d5 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:34.635893 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6211c4-e776-45b0-8e71-8ac5b18fd7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:34.640428 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6211c4-e776-45b0-8e71-8ac5b18fd7c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:34.640942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6211c4-e776-45b0-8e71-8ac5b18fd7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:34.642000 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6211c4-e776-45b0-8e71-8ac5b18fd7c3 admin admin] PATH_INFO: `/v3/users/b598b69364134e64a6e1c9e9fc140880` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:34.643925 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6211c4-e776-45b0-8e71-8ac5b18fd7c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b598b69364134e64a6e1c9e9fc140880)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:34.646100 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6211c4-e776-45b0-8e71-8ac5b18fd7c3 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:34.749159 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8e6211c4-e776-45b0-8e71-8ac5b18fd7c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b598b69364134e64a6e1c9e9fc140880'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:34.775907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8e6211c4-e776-45b0-8e71-8ac5b18fd7c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b598b69364134e64a6e1c9e9fc140880'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:34.788860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8e6211c4-e776-45b0-8e71-8ac5b18fd7c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b598b69364134e64a6e1c9e9fc140880'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:34.808697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8e6211c4-e776-45b0-8e71-8ac5b18fd7c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b598b69364134e64a6e1c9e9fc140880'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:34.809257 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 885/1795] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jun 22 11:45:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:45:34.811135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40592b9b-819d-4360-af2c-438f92fd39b4 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:34.812390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8e6211c4-e776-45b0-8e71-8ac5b18fd7c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b598b69364134e64a6e1c9e9fc140880'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:34.837757 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40592b9b-819d-4360-af2c-438f92fd39b4 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:34.838872 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 911/1796] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:34 2022] DELETE /identity/v3/users/b598b69364134e64a6e1c9e9fc140880 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:34.843435 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f24f6-fe34-4828-9d46-7395c4de85c4 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:34.843942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f24f6-fe34-4828-9d46-7395c4de85c4 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:34.844762 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f24f6-fe34-4828-9d46-7395c4de85c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:34.854313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:34.904147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-5a9f24f6-fe34-4828-9d46-7395c4de85c4 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:34.905910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5a9f24f6-fe34-4828-9d46-7395c4de85c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:35.035420 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a058d-4fd4-4e60-bb06-82da7a37b598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:35.036591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a058d-4fd4-4e60-bb06-82da7a37b598 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:35.043584 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a058d-4fd4-4e60-bb06-82da7a37b598 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:35.044148 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a058d-4fd4-4e60-bb06-82da7a37b598 admin admin] PATH_INFO: `/v3/projects/6db74dbfceeb49d88746eee3cf4c8ed0/users/466acb9d055c43fca1a856c976c267ab/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:35.082798 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 912/1797] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:35.106192 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a058d-4fd4-4e60-bb06-82da7a37b598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6db74dbfceeb49d88746eee3cf4c8ed0, user_id=466acb9d055c43fca1a856c976c267ab, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:35.117656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a058d-4fd4-4e60-bb06-82da7a37b598 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:35.129441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f24f6-fe34-4828-9d46-7395c4de85c4 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:35.131209 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5a9f24f6-fe34-4828-9d46-7395c4de85c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:35.167589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 886/1798] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:34 2022] PUT /identity/v3/projects/6db74dbfceeb49d88746eee3cf4c8ed0/users/466acb9d055c43fca1a856c976c267ab/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:35.171742 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a058d-4fd4-4e60-bb06-82da7a37b598 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:35.183495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-036a058d-4fd4-4e60-bb06-82da7a37b598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:35.307399 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f24f6-fe34-4828-9d46-7395c4de85c4 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:35.326686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a058d-4fd4-4e60-bb06-82da7a37b598 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:35.336983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456442d-a5d2-4c63-9569-2a56769bff8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:35.337856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456442d-a5d2-4c63-9569-2a56769bff8b admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:35.338104 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456442d-a5d2-4c63-9569-2a56769bff8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:35.338631 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b456442d-a5d2-4c63-9569-2a56769bff8b admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:35.340235 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b456442d-a5d2-4c63-9569-2a56769bff8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:35.341068 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b456442d-a5d2-4c63-9569-2a56769bff8b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:35.364813 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 887/1799] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:35.368440 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:35.369013 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456442d-a5d2-4c63-9569-2a56769bff8b admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:35.393919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b456442d-a5d2-4c63-9569-2a56769bff8b admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:35.413538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcaf06a-31cd-43eb-9ebf-53b949075865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:35.414444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcaf06a-31cd-43eb-9ebf-53b949075865 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:35.414735 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcaf06a-31cd-43eb-9ebf-53b949075865 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:35.415026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcaf06a-31cd-43eb-9ebf-53b949075865 admin admin] PATH_INFO: `/v3/projects/6db74dbfceeb49d88746eee3cf4c8ed0/users/466acb9d055c43fca1a856c976c267ab/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:35.424283 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcaf06a-31cd-43eb-9ebf-53b949075865 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6db74dbfceeb49d88746eee3cf4c8ed0, user_id=466acb9d055c43fca1a856c976c267ab, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:35.426976 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcaf06a-31cd-43eb-9ebf-53b949075865 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:35.470210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 888/1800] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:35 2022] PUT /identity/v3/projects/6db74dbfceeb49d88746eee3cf4c8ed0/users/466acb9d055c43fca1a856c976c267ab/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:45:35.476002 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b0b3d-23ea-4f6e-b48f-00ffa7e5dab6 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:35.476296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b0b3d-23ea-4f6e-b48f-00ffa7e5dab6 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:35.476580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b0b3d-23ea-4f6e-b48f-00ffa7e5dab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:35.487393 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bf9fc-1cc1-4910-9a04-6360358b03a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:35.489895 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bf9fc-1cc1-4910-9a04-6360358b03a3 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:35.490867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bf9fc-1cc1-4910-9a04-6360358b03a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:35.491449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bf9fc-1cc1-4910-9a04-6360358b03a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:35.520245 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55bf9fc-1cc1-4910-9a04-6360358b03a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:35.531707 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55bf9fc-1cc1-4910-9a04-6360358b03a3 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:35.540499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-dc7b0b3d-23ea-4f6e-b48f-00ffa7e5dab6 None None] MFA Rules not processed for user `466acb9d055c43fca1a856c976c267ab`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:35.543850 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-dc7b0b3d-23ea-4f6e-b48f-00ffa7e5dab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:35.585135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 913/1801] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:35.674259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bf9fc-1cc1-4910-9a04-6360358b03a3 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:35.676255 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b55bf9fc-1cc1-4910-9a04-6360358b03a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:35.767617 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:35.778132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 889/1802] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:35.791789 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b0b3d-23ea-4f6e-b48f-00ffa7e5dab6 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:35.793416 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-dc7b0b3d-23ea-4f6e-b48f-00ffa7e5dab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:35.902382 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bf9fc-1cc1-4910-9a04-6360358b03a3 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:35.919725 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b0b3d-23ea-4f6e-b48f-00ffa7e5dab6 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:35.926126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63678a71-a88c-45d0-96f0-13427668960a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:35.926921 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63678a71-a88c-45d0-96f0-13427668960a service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:35.927302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63678a71-a88c-45d0-96f0-13427668960a service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:35.927730 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63678a71-a88c-45d0-96f0-13427668960a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:35.944180 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63678a71-a88c-45d0-96f0-13427668960a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:35.947276 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63678a71-a88c-45d0-96f0-13427668960a service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:35.964413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 890/1803] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:36.053141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde9137-5364-4759-9d39-749c52cee9c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:36.053818 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde9137-5364-4759-9d39-749c52cee9c1 service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:36.054104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde9137-5364-4759-9d39-749c52cee9c1 service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:36.054528 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde9137-5364-4759-9d39-749c52cee9c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:36.073682 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-0fde9137-5364-4759-9d39-749c52cee9c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:36.107408 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde9137-5364-4759-9d39-749c52cee9c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:36.109577 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde9137-5364-4759-9d39-749c52cee9c1 service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:36.162169 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 914/1804] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:36.549026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63678a71-a88c-45d0-96f0-13427668960a service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:36.551851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-63678a71-a88c-45d0-96f0-13427668960a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:36.681388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63678a71-a88c-45d0-96f0-13427668960a service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:36.706629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f9341-d0ee-4c10-8048-613e701418a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:36.707708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f9341-d0ee-4c10-8048-613e701418a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:36.709106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f9341-d0ee-4c10-8048-613e701418a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:36.712166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f9341-d0ee-4c10-8048-613e701418a2 admin admin] PATH_INFO: `/v3/projects/c650c8e4d52f4d448aeac0e541ceec6b` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:36.720805 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f9341-d0ee-4c10-8048-613e701418a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c650c8e4d52f4d448aeac0e541ceec6b)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:36.727590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f9341-d0ee-4c10-8048-613e701418a2 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:36.892500 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-e06f9341-d0ee-4c10-8048-613e701418a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c650c8e4d52f4d448aeac0e541ceec6b', 'request_id': 'req-e06f9341-d0ee-4c10-8048-613e701418a2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:36.908926 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-e06f9341-d0ee-4c10-8048-613e701418a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c650c8e4d52f4d448aeac0e541ceec6b', 'request_id': 'req-e06f9341-d0ee-4c10-8048-613e701418a2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:36.924895 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-e06f9341-d0ee-4c10-8048-613e701418a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c650c8e4d52f4d448aeac0e541ceec6b', 'request_id': 'req-e06f9341-d0ee-4c10-8048-613e701418a2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:36.956772 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-e06f9341-d0ee-4c10-8048-613e701418a2 admin admin] The token cache is being invalidate because project c650c8e4d52f4d448aeac0e541ceec6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:36.957083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-e06f9341-d0ee-4c10-8048-613e701418a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:36.961066 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-e06f9341-d0ee-4c10-8048-613e701418a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:36.962948 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 891/1805] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:36 2022] DELETE /identity/v3/projects/c650c8e4d52f4d448aeac0e541ceec6b => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:36.967845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35882760-347d-4ab7-820c-a86acb8981b8 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:36.968989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35882760-347d-4ab7-820c-a86acb8981b8 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:36.969444 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35882760-347d-4ab7-820c-a86acb8981b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:37.018560 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-35882760-347d-4ab7-820c-a86acb8981b8 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:37.024474 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-35882760-347d-4ab7-820c-a86acb8981b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:37.174429 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:37.215067 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 915/1806] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:37.224323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f3716-0ecb-41a9-82ff-965a9e9c2008 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:37.224922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f3716-0ecb-41a9-82ff-965a9e9c2008 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:37.225403 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f3716-0ecb-41a9-82ff-965a9e9c2008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:37.279071 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-737f3716-0ecb-41a9-82ff-965a9e9c2008 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:37.280861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-737f3716-0ecb-41a9-82ff-965a9e9c2008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:37.341715 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 892/1807] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:37.346912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35882760-347d-4ab7-820c-a86acb8981b8 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:37.375097 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35882760-347d-4ab7-820c-a86acb8981b8 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:37.428068 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83eeb9e-20c5-46fc-9f23-689d9f3e5c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:37.429237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83eeb9e-20c5-46fc-9f23-689d9f3e5c3a admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:37.429644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83eeb9e-20c5-46fc-9f23-689d9f3e5c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:37.430107 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83eeb9e-20c5-46fc-9f23-689d9f3e5c3a admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:37.440987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83eeb9e-20c5-46fc-9f23-689d9f3e5c3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:37.448123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83eeb9e-20c5-46fc-9f23-689d9f3e5c3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:37.448123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83eeb9e-20c5-46fc-9f23-689d9f3e5c3a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:37.464232 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f3716-0ecb-41a9-82ff-965a9e9c2008 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:37.468241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-737f3716-0ecb-41a9-82ff-965a9e9c2008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:37.516615 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 916/1808] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:37.520708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83eeb9e-20c5-46fc-9f23-689d9f3e5c3a admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:37.593556 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83eeb9e-20c5-46fc-9f23-689d9f3e5c3a admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:37.629847 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a418d7-fed7-49c4-8951-548401986533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:37.631738 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a418d7-fed7-49c4-8951-548401986533 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:37.632351 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a418d7-fed7-49c4-8951-548401986533 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:37.632664 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a418d7-fed7-49c4-8951-548401986533 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:37.633683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a418d7-fed7-49c4-8951-548401986533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:37.636088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a418d7-fed7-49c4-8951-548401986533 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:37.692712 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:37.738322 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 917/1809] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:37 2022] POST /identity/v3/projects => generated 379 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:37.741372 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a418d7-fed7-49c4-8951-548401986533 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:37.778533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f3716-0ecb-41a9-82ff-965a9e9c2008 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:37.952245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beab4b2-7d24-44d0-8050-8773f87af41c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:37.956315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beab4b2-7d24-44d0-8050-8773f87af41c service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:37.957388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beab4b2-7d24-44d0-8050-8773f87af41c service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:37.957685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beab4b2-7d24-44d0-8050-8773f87af41c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:37.976108 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-7beab4b2-7d24-44d0-8050-8773f87af41c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:38.006656 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a418d7-fed7-49c4-8951-548401986533 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:38.035328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f8cdd-56f9-42d9-b4a6-0170ed558fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:38.042765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f8cdd-56f9-42d9-b4a6-0170ed558fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:38.044400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f8cdd-56f9-42d9-b4a6-0170ed558fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:38.045798 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f8cdd-56f9-42d9-b4a6-0170ed558fc9 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:38.047142 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f8cdd-56f9-42d9-b4a6-0170ed558fc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:38.062313 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f8cdd-56f9-42d9-b4a6-0170ed558fc9 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:38.146148 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 918/1810] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:37 2022] POST /identity/v3/users => generated 290 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:38.148373 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f8cdd-56f9-42d9-b4a6-0170ed558fc9 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:38.153368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e00f8cdd-56f9-42d9-b4a6-0170ed558fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:38.170516 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beab4b2-7d24-44d0-8050-8773f87af41c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:38.176335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beab4b2-7d24-44d0-8050-8773f87af41c service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:38.198047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f8cdd-56f9-42d9-b4a6-0170ed558fc9 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:38.218802 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624f85d-9f9f-4948-9b3a-52dec4d183c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:38.221128 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624f85d-9f9f-4948-9b3a-52dec4d183c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:38.221480 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624f85d-9f9f-4948-9b3a-52dec4d183c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:38.223152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624f85d-9f9f-4948-9b3a-52dec4d183c4 admin admin] PATH_INFO: `/v3/users/4087a6b4a4024774b3f0961406ee5222` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:38.227766 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624f85d-9f9f-4948-9b3a-52dec4d183c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4087a6b4a4024774b3f0961406ee5222)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:38.231097 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624f85d-9f9f-4948-9b3a-52dec4d183c4 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:38.231952 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 893/1811] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 773 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:38.233388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beab4b2-7d24-44d0-8050-8773f87af41c service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:38.301891 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beab4b2-7d24-44d0-8050-8773f87af41c service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:38.332870 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae58c4a-c14a-4980-9442-017bef250a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:38.333922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae58c4a-c14a-4980-9442-017bef250a2e admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:38.334359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae58c4a-c14a-4980-9442-017bef250a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:38.334782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae58c4a-c14a-4980-9442-017bef250a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:38.336002 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae58c4a-c14a-4980-9442-017bef250a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:38.339636 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae58c4a-c14a-4980-9442-017bef250a2e admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:38.348604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3624f85d-9f9f-4948-9b3a-52dec4d183c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4087a6b4a4024774b3f0961406ee5222'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:38.385007 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3624f85d-9f9f-4948-9b3a-52dec4d183c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4087a6b4a4024774b3f0961406ee5222'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:38.396883 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 894/1812] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:38.398452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae58c4a-c14a-4980-9442-017bef250a2e admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:38.399985 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3624f85d-9f9f-4948-9b3a-52dec4d183c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4087a6b4a4024774b3f0961406ee5222'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:38.438102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3624f85d-9f9f-4948-9b3a-52dec4d183c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4087a6b4a4024774b3f0961406ee5222'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:38.440426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3624f85d-9f9f-4948-9b3a-52dec4d183c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4087a6b4a4024774b3f0961406ee5222'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:38.448914 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae58c4a-c14a-4980-9442-017bef250a2e admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:38.458851 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 919/1813] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:38 2022] DELETE /identity/v3/users/4087a6b4a4024774b3f0961406ee5222 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:38.463708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3b056-2039-4577-8719-72724b1cef3b None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:38.464074 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3b056-2039-4577-8719-72724b1cef3b None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:38.464476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3b056-2039-4577-8719-72724b1cef3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:38.546734 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-18c3b056-2039-4577-8719-72724b1cef3b None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:38.552848 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-18c3b056-2039-4577-8719-72724b1cef3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:38.622069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:38.658953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50fe09-3f2e-4790-bc5c-cb08e3be8bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:38.659806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50fe09-3f2e-4790-bc5c-cb08e3be8bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:38.661451 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50fe09-3f2e-4790-bc5c-cb08e3be8bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:38.662747 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50fe09-3f2e-4790-bc5c-cb08e3be8bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:38.664178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50fe09-3f2e-4790-bc5c-cb08e3be8bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:38.670947 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50fe09-3f2e-4790-bc5c-cb08e3be8bc1 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:38.700363 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 895/1814] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:38.722004 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50fe09-3f2e-4790-bc5c-cb08e3be8bc1 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:38.730140 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-bd50fe09-3f2e-4790-bc5c-cb08e3be8bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:38.772206 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50fe09-3f2e-4790-bc5c-cb08e3be8bc1 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:38.778237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 920/1815] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:38.786747 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c6041-c51d-445e-91ec-c33e2c36fcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:38.789079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c6041-c51d-445e-91ec-c33e2c36fcb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:38.790604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c6041-c51d-445e-91ec-c33e2c36fcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:38.792031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c6041-c51d-445e-91ec-c33e2c36fcb7 admin admin] PATH_INFO: `/v3/projects/ebd16beaed7747968b3afee4e9d0af19/users/0b8e75d9eeef4a419af5d7f8d06234b5/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:38.826420 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3b056-2039-4577-8719-72724b1cef3b None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:38.828407 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-18c3b056-2039-4577-8719-72724b1cef3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:38.834948 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c6041-c51d-445e-91ec-c33e2c36fcb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebd16beaed7747968b3afee4e9d0af19, user_id=0b8e75d9eeef4a419af5d7f8d06234b5, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:38.838420 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c6041-c51d-445e-91ec-c33e2c36fcb7 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:38.856161 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:38.867864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 896/1816] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:38 2022] PUT /identity/v3/projects/ebd16beaed7747968b3afee4e9d0af19/users/0b8e75d9eeef4a419af5d7f8d06234b5/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:38.878284 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71e424-35cf-441c-9804-201f89d6a020 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:38.878716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71e424-35cf-441c-9804-201f89d6a020 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:38.879094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e71e424-35cf-441c-9804-201f89d6a020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:38.957575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-8e71e424-35cf-441c-9804-201f89d6a020 None None] MFA Rules not processed for user `0b8e75d9eeef4a419af5d7f8d06234b5`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:38.960231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-8e71e424-35cf-441c-9804-201f89d6a020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:39.052938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3b056-2039-4577-8719-72724b1cef3b None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:39.175921 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 897/1817] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:39.181170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71e424-35cf-441c-9804-201f89d6a020 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:39.194679 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303683c-b9b9-4d73-981f-84eefe7ad073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:39.195887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303683c-b9b9-4d73-981f-84eefe7ad073 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:39.196439 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303683c-b9b9-4d73-981f-84eefe7ad073 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:39.197040 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303683c-b9b9-4d73-981f-84eefe7ad073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:39.361982 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303683c-b9b9-4d73-981f-84eefe7ad073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:39.371491 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303683c-b9b9-4d73-981f-84eefe7ad073 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:39.401991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e71e424-35cf-441c-9804-201f89d6a020 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:39.408135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 921/1818] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:45:39.408389 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425b7f3-cfff-426d-a4f8-a17c23a1aa4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:39.412311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425b7f3-cfff-426d-a4f8-a17c23a1aa4c admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:39.412921 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425b7f3-cfff-426d-a4f8-a17c23a1aa4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:39.413987 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425b7f3-cfff-426d-a4f8-a17c23a1aa4c admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:39.417430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d425b7f3-cfff-426d-a4f8-a17c23a1aa4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:39.421784 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d425b7f3-cfff-426d-a4f8-a17c23a1aa4c admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:39.483138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 898/1819] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:39 2022] POST /identity/v3/projects => generated 381 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:39.485961 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303683c-b9b9-4d73-981f-84eefe7ad073 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:39.499607 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303683c-b9b9-4d73-981f-84eefe7ad073 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:39.546590 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:39.664037 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e9465-44f2-49ba-a436-bf7616ee5cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:39.664924 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e9465-44f2-49ba-a436-bf7616ee5cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:39.665308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e9465-44f2-49ba-a436-bf7616ee5cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:39.665750 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e9465-44f2-49ba-a436-bf7616ee5cf8 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:39.671228 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e9465-44f2-49ba-a436-bf7616ee5cf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:39.681042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e9465-44f2-49ba-a436-bf7616ee5cf8 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:39.737500 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425b7f3-cfff-426d-a4f8-a17c23a1aa4c admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:39.747611 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 922/1820] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:39 2022] POST /identity/v3/users => generated 297 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:39.767448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425b7f3-cfff-426d-a4f8-a17c23a1aa4c admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:39.783317 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e9465-44f2-49ba-a436-bf7616ee5cf8 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:39.785034 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-3f2e9465-44f2-49ba-a436-bf7616ee5cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:39.804176 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743ae5c-1174-410e-b570-9b9b2140d9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:39.807189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743ae5c-1174-410e-b570-9b9b2140d9ef admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:39.807189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743ae5c-1174-410e-b570-9b9b2140d9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:39.807189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743ae5c-1174-410e-b570-9b9b2140d9ef admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:39.807724 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1743ae5c-1174-410e-b570-9b9b2140d9ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:39.810694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e9465-44f2-49ba-a436-bf7616ee5cf8 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:39.824406 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1743ae5c-1174-410e-b570-9b9b2140d9ef admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:39.830564 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623471af-71b5-466a-81a2-440786dad3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:39.831493 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623471af-71b5-466a-81a2-440786dad3dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:39.831779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623471af-71b5-466a-81a2-440786dad3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:39.832057 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623471af-71b5-466a-81a2-440786dad3dd admin admin] PATH_INFO: `/v3/projects/4037b91404bd493095b736a4f8db43c5` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:39.839543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623471af-71b5-466a-81a2-440786dad3dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4037b91404bd493095b736a4f8db43c5)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:39.841603 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623471af-71b5-466a-81a2-440786dad3dd admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:39.868812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 899/1821] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:39.877920 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743ae5c-1174-410e-b570-9b9b2140d9ef admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:39.915366 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743ae5c-1174-410e-b570-9b9b2140d9ef admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:39.954385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b69fa0-8897-451e-ae70-a000a60a0089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:39.956981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b69fa0-8897-451e-ae70-a000a60a0089 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:39.957396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b69fa0-8897-451e-ae70-a000a60a0089 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:39.959484 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b69fa0-8897-451e-ae70-a000a60a0089 admin admin] PATH_INFO: `/v3/projects/c30c5caee2f249a28b1eb4398c3e4d19/users/de950003de87400f8f94a31e3ea23110/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:39.990812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-623471af-71b5-466a-81a2-440786dad3dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4037b91404bd493095b736a4f8db43c5', 'request_id': 'req-623471af-71b5-466a-81a2-440786dad3dd'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:40.002195 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-623471af-71b5-466a-81a2-440786dad3dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4037b91404bd493095b736a4f8db43c5', 'request_id': 'req-623471af-71b5-466a-81a2-440786dad3dd'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:40.030733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-623471af-71b5-466a-81a2-440786dad3dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4037b91404bd493095b736a4f8db43c5', 'request_id': 'req-623471af-71b5-466a-81a2-440786dad3dd'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:40.032420 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b69fa0-8897-451e-ae70-a000a60a0089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c30c5caee2f249a28b1eb4398c3e4d19, user_id=de950003de87400f8f94a31e3ea23110, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:40.040077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b69fa0-8897-451e-ae70-a000a60a0089 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:40.061290 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-623471af-71b5-466a-81a2-440786dad3dd admin admin] The token cache is being invalidate because project 4037b91404bd493095b736a4f8db43c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:40.063644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-623471af-71b5-466a-81a2-440786dad3dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:40.065120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-623471af-71b5-466a-81a2-440786dad3dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:40.069938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 923/1822] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:39 2022] DELETE /identity/v3/projects/4037b91404bd493095b736a4f8db43c5 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:40.073618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623471af-71b5-466a-81a2-440786dad3dd admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:40.075039 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 900/1823] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:39 2022] PUT /identity/v3/projects/c30c5caee2f249a28b1eb4398c3e4d19/users/de950003de87400f8f94a31e3ea23110/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:40.076402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-623471af-71b5-466a-81a2-440786dad3dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:40.082273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b69fa0-8897-451e-ae70-a000a60a0089 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:40.086572 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-93b69fa0-8897-451e-ae70-a000a60a0089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:40.213805 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:40.248045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b69fa0-8897-451e-ae70-a000a60a0089 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:40.258623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7fd24-5fa2-4a45-85ad-cb43e7db6c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:40.259982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7fd24-5fa2-4a45-85ad-cb43e7db6c93 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:40.260298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7fd24-5fa2-4a45-85ad-cb43e7db6c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:40.260652 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7fd24-5fa2-4a45-85ad-cb43e7db6c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:40.264010 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7fd24-5fa2-4a45-85ad-cb43e7db6c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:40.266356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7fd24-5fa2-4a45-85ad-cb43e7db6c93 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:40.314350 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 901/1824] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:40.315531 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7fd24-5fa2-4a45-85ad-cb43e7db6c93 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:40.338955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623471af-71b5-466a-81a2-440786dad3dd admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:40.348847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7fd24-5fa2-4a45-85ad-cb43e7db6c93 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:40.362790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278564d-3b5f-443c-8c4e-cdc45eebf22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:40.366453 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f19b3-4ba0-4043-94a2-8aa7dd61d996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:40.367557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f19b3-4ba0-4043-94a2-8aa7dd61d996 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:40.367854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f19b3-4ba0-4043-94a2-8aa7dd61d996 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:40.368138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f19b3-4ba0-4043-94a2-8aa7dd61d996 admin admin] PATH_INFO: `/v3/projects/c30c5caee2f249a28b1eb4398c3e4d19/users/de950003de87400f8f94a31e3ea23110/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:40.370155 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278564d-3b5f-443c-8c4e-cdc45eebf22f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:40.371549 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278564d-3b5f-443c-8c4e-cdc45eebf22f admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:40.371843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278564d-3b5f-443c-8c4e-cdc45eebf22f admin admin] PATH_INFO: `/v3/users/1d360d4ce5dd4237bdb7154ec5dba559` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:40.372307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f19b3-4ba0-4043-94a2-8aa7dd61d996 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c30c5caee2f249a28b1eb4398c3e4d19, user_id=de950003de87400f8f94a31e3ea23110, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:40.376993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7278564d-3b5f-443c-8c4e-cdc45eebf22f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d360d4ce5dd4237bdb7154ec5dba559)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:40.379291 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7278564d-3b5f-443c-8c4e-cdc45eebf22f admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:40.383913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f19b3-4ba0-4043-94a2-8aa7dd61d996 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:40.425962 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 902/1825] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:40 2022] PUT /identity/v3/projects/c30c5caee2f249a28b1eb4398c3e4d19/users/de950003de87400f8f94a31e3ea23110/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:40.431409 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca60ba-0e2c-4610-b4d2-aeed6a557a06 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:40.431760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca60ba-0e2c-4610-b4d2-aeed6a557a06 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:40.432119 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca60ba-0e2c-4610-b4d2-aeed6a557a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:40.477274 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:40.489871 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-7278564d-3b5f-443c-8c4e-cdc45eebf22f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d360d4ce5dd4237bdb7154ec5dba559'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:40.505387 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-f0ca60ba-0e2c-4610-b4d2-aeed6a557a06 None None] MFA Rules not processed for user `de950003de87400f8f94a31e3ea23110`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:40.506692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f0ca60ba-0e2c-4610-b4d2-aeed6a557a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:40.522207 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-7278564d-3b5f-443c-8c4e-cdc45eebf22f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d360d4ce5dd4237bdb7154ec5dba559'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:40.541047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-7278564d-3b5f-443c-8c4e-cdc45eebf22f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d360d4ce5dd4237bdb7154ec5dba559'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:40.557425 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-7278564d-3b5f-443c-8c4e-cdc45eebf22f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d360d4ce5dd4237bdb7154ec5dba559'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:40.558669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-7278564d-3b5f-443c-8c4e-cdc45eebf22f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d360d4ce5dd4237bdb7154ec5dba559'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:40.584709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 924/1826] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:40 2022] DELETE /identity/v3/users/1d360d4ce5dd4237bdb7154ec5dba559 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:40.739912 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 903/1827] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:40.931621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2dee2-5981-481c-82c3-4774edaf48a9 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:40.931621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2dee2-5981-481c-82c3-4774edaf48a9 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:40.931621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2dee2-5981-481c-82c3-4774edaf48a9 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:40.931621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 925/1828] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jun 22 11:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:45:40.943233 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7050f29-807e-4053-8a19-50dcf4844ea7 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:40.943764 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7050f29-807e-4053-8a19-50dcf4844ea7 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:40.944212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7050f29-807e-4053-8a19-50dcf4844ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:40.984550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c7050f29-807e-4053-8a19-50dcf4844ea7 None None] MFA Rules not processed for user `479a376dcac14c509c1c55f0046d88cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:40.986331 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c7050f29-807e-4053-8a19-50dcf4844ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:41.129631 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 904/1829] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jun 22 11:45:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:45:41.383170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2dee2-5981-481c-82c3-4774edaf48a9 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:41.385726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-50f2dee2-5981-481c-82c3-4774edaf48a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:41.434172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2dee2-5981-481c-82c3-4774edaf48a9 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:41.448463 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2fb72d-de77-443a-93a1-6da7760849e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:41.448463 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fb72d-de77-443a-93a1-6da7760849e3 service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:41.448463 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fb72d-de77-443a-93a1-6da7760849e3 service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:41.450420 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fb72d-de77-443a-93a1-6da7760849e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:41.476924 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2fb72d-de77-443a-93a1-6da7760849e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:41.485304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2fb72d-de77-443a-93a1-6da7760849e3 service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:41.522273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 926/1830] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:41.545217 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7050f29-807e-4053-8a19-50dcf4844ea7 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:41.546748 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c7050f29-807e-4053-8a19-50dcf4844ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:41.553788 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2fb72d-de77-443a-93a1-6da7760849e3 service nova] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:41.555370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-be2fb72d-de77-443a-93a1-6da7760849e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:41.659768 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:41.737111 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7050f29-807e-4053-8a19-50dcf4844ea7 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:41.752154 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480a60d-cc6e-4c7f-a681-28784ae763b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:41.752557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2fb72d-de77-443a-93a1-6da7760849e3 service nova] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:41.753555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480a60d-cc6e-4c7f-a681-28784ae763b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:41.754193 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480a60d-cc6e-4c7f-a681-28784ae763b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:41.754941 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480a60d-cc6e-4c7f-a681-28784ae763b1 admin admin] PATH_INFO: `/v3/projects/9d344db5c7a9431cad3e975c128ce7d9` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:41.756831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480a60d-cc6e-4c7f-a681-28784ae763b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d344db5c7a9431cad3e975c128ce7d9)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:41.760214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480a60d-cc6e-4c7f-a681-28784ae763b1 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:41.761180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75958ea8-b8ec-4dba-a2af-5d99ae6e0404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:41.762712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75958ea8-b8ec-4dba-a2af-5d99ae6e0404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:41.763059 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75958ea8-b8ec-4dba-a2af-5d99ae6e0404 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:41.763343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75958ea8-b8ec-4dba-a2af-5d99ae6e0404 admin admin] PATH_INFO: `/v3/projects/c1d90ace493343819f5874743800f570` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:41.765034 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75958ea8-b8ec-4dba-a2af-5d99ae6e0404 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1d90ace493343819f5874743800f570)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:41.769930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75958ea8-b8ec-4dba-a2af-5d99ae6e0404 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:41.859387 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-2480a60d-cc6e-4c7f-a681-28784ae763b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d344db5c7a9431cad3e975c128ce7d9', 'request_id': 'req-2480a60d-cc6e-4c7f-a681-28784ae763b1'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:41.865933 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-75958ea8-b8ec-4dba-a2af-5d99ae6e0404 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1d90ace493343819f5874743800f570', 'request_id': 'req-75958ea8-b8ec-4dba-a2af-5d99ae6e0404'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:41.871223 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-2480a60d-cc6e-4c7f-a681-28784ae763b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d344db5c7a9431cad3e975c128ce7d9', 'request_id': 'req-2480a60d-cc6e-4c7f-a681-28784ae763b1'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:41.884402 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-2480a60d-cc6e-4c7f-a681-28784ae763b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d344db5c7a9431cad3e975c128ce7d9', 'request_id': 'req-2480a60d-cc6e-4c7f-a681-28784ae763b1'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:41.888265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-75958ea8-b8ec-4dba-a2af-5d99ae6e0404 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1d90ace493343819f5874743800f570', 'request_id': 'req-75958ea8-b8ec-4dba-a2af-5d99ae6e0404'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:41.903127 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-2480a60d-cc6e-4c7f-a681-28784ae763b1 admin admin] The token cache is being invalidate because project 9d344db5c7a9431cad3e975c128ce7d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:41.903674 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-2480a60d-cc6e-4c7f-a681-28784ae763b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:41.904841 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-2480a60d-cc6e-4c7f-a681-28784ae763b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:41.908501 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-75958ea8-b8ec-4dba-a2af-5d99ae6e0404 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1d90ace493343819f5874743800f570', 'request_id': 'req-75958ea8-b8ec-4dba-a2af-5d99ae6e0404'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:41.913948 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 905/1831] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:41 2022] DELETE /identity/v3/projects/9d344db5c7a9431cad3e975c128ce7d9 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:45:41.922773 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480a60d-cc6e-4c7f-a681-28784ae763b1 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:41.926338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2480a60d-cc6e-4c7f-a681-28784ae763b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:41.939175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-75958ea8-b8ec-4dba-a2af-5d99ae6e0404 admin admin] The token cache is being invalidate because project c1d90ace493343819f5874743800f570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:41.939705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-75958ea8-b8ec-4dba-a2af-5d99ae6e0404 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:41.941108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-75958ea8-b8ec-4dba-a2af-5d99ae6e0404 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:41.943396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 927/1832] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:41 2022] DELETE /identity/v3/projects/c1d90ace493343819f5874743800f570 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:45:41.950254 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42447082-4fa8-4e51-8db1-9d7766fbd000 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:41.950936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42447082-4fa8-4e51-8db1-9d7766fbd000 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:41.952011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42447082-4fa8-4e51-8db1-9d7766fbd000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:41.990360 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-42447082-4fa8-4e51-8db1-9d7766fbd000 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:41.991905 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-42447082-4fa8-4e51-8db1-9d7766fbd000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:42.060987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:42.080271 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480a60d-cc6e-4c7f-a681-28784ae763b1 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:42.136673 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 928/1833] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:42.137570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42447082-4fa8-4e51-8db1-9d7766fbd000 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:42.142654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-42447082-4fa8-4e51-8db1-9d7766fbd000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:42.169277 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42447082-4fa8-4e51-8db1-9d7766fbd000 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:42.175381 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754915b-5f07-4227-b1c6-fa762d7ad4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:42.176152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754915b-5f07-4227-b1c6-fa762d7ad4d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:42.176369 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754915b-5f07-4227-b1c6-fa762d7ad4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:42.176633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754915b-5f07-4227-b1c6-fa762d7ad4d9 admin admin] PATH_INFO: `/v3/users/7859085336694721a5cc7a70c8cc0d64` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:42.177907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754915b-5f07-4227-b1c6-fa762d7ad4d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7859085336694721a5cc7a70c8cc0d64)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:42.179996 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754915b-5f07-4227-b1c6-fa762d7ad4d9 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:42.210148 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9229e-d47b-40aa-9606-c6fdad9726aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:42.212537 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9229e-d47b-40aa-9606-c6fdad9726aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:42.213043 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9229e-d47b-40aa-9606-c6fdad9726aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:42.213544 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9229e-d47b-40aa-9606-c6fdad9726aa admin admin] PATH_INFO: `/v3/users/466acb9d055c43fca1a856c976c267ab` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:42.215891 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9229e-d47b-40aa-9606-c6fdad9726aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=466acb9d055c43fca1a856c976c267ab)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:42.218644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9229e-d47b-40aa-9606-c6fdad9726aa admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:42.253831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:42.292340 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f754915b-5f07-4227-b1c6-fa762d7ad4d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7859085336694721a5cc7a70c8cc0d64'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:42.305130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a8f9229e-d47b-40aa-9606-c6fdad9726aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '466acb9d055c43fca1a856c976c267ab'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:42.319975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f754915b-5f07-4227-b1c6-fa762d7ad4d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7859085336694721a5cc7a70c8cc0d64'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:42.344045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f754915b-5f07-4227-b1c6-fa762d7ad4d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7859085336694721a5cc7a70c8cc0d64'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:42.345545 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a8f9229e-d47b-40aa-9606-c6fdad9726aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '466acb9d055c43fca1a856c976c267ab'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:42.358376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a8f9229e-d47b-40aa-9606-c6fdad9726aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '466acb9d055c43fca1a856c976c267ab'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:42.359150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a8f9229e-d47b-40aa-9606-c6fdad9726aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '466acb9d055c43fca1a856c976c267ab'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:42.367849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f754915b-5f07-4227-b1c6-fa762d7ad4d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7859085336694721a5cc7a70c8cc0d64'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:42.369268 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f754915b-5f07-4227-b1c6-fa762d7ad4d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7859085336694721a5cc7a70c8cc0d64'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:42.376741 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a8f9229e-d47b-40aa-9606-c6fdad9726aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '466acb9d055c43fca1a856c976c267ab'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:42.392760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 906/1834] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:41 2022] DELETE /identity/v3/users/466acb9d055c43fca1a856c976c267ab => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:42.399743 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9229e-d47b-40aa-9606-c6fdad9726aa admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:42.401645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 929/1835] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:42 2022] DELETE /identity/v3/users/7859085336694721a5cc7a70c8cc0d64 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:42.403533 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656242e3-b4d4-4654-8eb9-0dfeaa5be81b None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:42.403834 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656242e3-b4d4-4654-8eb9-0dfeaa5be81b None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:42.404119 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656242e3-b4d4-4654-8eb9-0dfeaa5be81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:42.404824 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a8f9229e-d47b-40aa-9606-c6fdad9726aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:42.442044 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-656242e3-b4d4-4654-8eb9-0dfeaa5be81b None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:42.446486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-656242e3-b4d4-4654-8eb9-0dfeaa5be81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:42.553333 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9229e-d47b-40aa-9606-c6fdad9726aa admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:42.587923 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 930/1836] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:42.592020 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cab43-8680-41ce-a300-2536f6f1af89 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:42.592575 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cab43-8680-41ce-a300-2536f6f1af89 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:42.593026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cab43-8680-41ce-a300-2536f6f1af89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:42.636935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-af6cab43-8680-41ce-a300-2536f6f1af89 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:42.639922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-af6cab43-8680-41ce-a300-2536f6f1af89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:42.665708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 931/1837] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:42.667843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cab43-8680-41ce-a300-2536f6f1af89 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:42.683924 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f10c39e-c653-49b9-a3d4-30a942e7c353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:42.684802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10c39e-c653-49b9-a3d4-30a942e7c353 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:42.685210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10c39e-c653-49b9-a3d4-30a942e7c353 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:42.686294 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10c39e-c653-49b9-a3d4-30a942e7c353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:42.688579 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cab43-8680-41ce-a300-2536f6f1af89 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:42.689218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-4f10c39e-c653-49b9-a3d4-30a942e7c353 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:42.696628 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acc017-62e3-4b9d-94d8-4a97c208e130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:42.698966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acc017-62e3-4b9d-94d8-4a97c208e130 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:42.700175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acc017-62e3-4b9d-94d8-4a97c208e130 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:42.700912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acc017-62e3-4b9d-94d8-4a97c208e130 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:42.702440 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97acc017-62e3-4b9d-94d8-4a97c208e130 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:42.702967 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97acc017-62e3-4b9d-94d8-4a97c208e130 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:42.706431 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97acc017-62e3-4b9d-94d8-4a97c208e130 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:42.727387 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 932/1838] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:42.729755 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acc017-62e3-4b9d-94d8-4a97c208e130 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:42.733171 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-97acc017-62e3-4b9d-94d8-4a97c208e130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:42.804157 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:42.819486 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f10c39e-c653-49b9-a3d4-30a942e7c353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:42.821068 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f10c39e-c653-49b9-a3d4-30a942e7c353 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:42.830647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:42.838021 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 907/1839] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:42.838893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f10c39e-c653-49b9-a3d4-30a942e7c353 service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:42.858594 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f10c39e-c653-49b9-a3d4-30a942e7c353 service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:42.870455 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a9467-de38-42f1-b52d-890c61a1360a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:42.871171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a9467-de38-42f1-b52d-890c61a1360a admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:42.871522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a9467-de38-42f1-b52d-890c61a1360a admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:42.871797 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a9467-de38-42f1-b52d-890c61a1360a admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:42.872491 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a9467-de38-42f1-b52d-890c61a1360a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:42.879625 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a9467-de38-42f1-b52d-890c61a1360a admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:42.888065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acc017-62e3-4b9d-94d8-4a97c208e130 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:42.920162 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28d8f7-c910-4b5d-9ba2-2b1495214ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:42.920834 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28d8f7-c910-4b5d-9ba2-2b1495214ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:42.921120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28d8f7-c910-4b5d-9ba2-2b1495214ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:42.921399 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28d8f7-c910-4b5d-9ba2-2b1495214ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:42.943902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 908/1840] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:42 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:42.943902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a9467-de38-42f1-b52d-890c61a1360a admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:42.944675 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28d8f7-c910-4b5d-9ba2-2b1495214ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:42.945688 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28d8f7-c910-4b5d-9ba2-2b1495214ef5 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:42.968274 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 933/1841] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:43.051022 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a9467-de38-42f1-b52d-890c61a1360a admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:43.058767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf7d56-9f2a-4b7e-9a9d-3d7f117031d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:43.060076 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7d56-9f2a-4b7e-9a9d-3d7f117031d6 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:43.060516 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7d56-9f2a-4b7e-9a9d-3d7f117031d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:43.061673 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7d56-9f2a-4b7e-9a9d-3d7f117031d6 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:43.063544 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf7d56-9f2a-4b7e-9a9d-3d7f117031d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:43.065803 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf7d56-9f2a-4b7e-9a9d-3d7f117031d6 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:43.101670 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 909/1842] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:42 2022] POST /identity/v3/users => generated 284 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:43.106297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28d8f7-c910-4b5d-9ba2-2b1495214ef5 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:43.144100 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28d8f7-c910-4b5d-9ba2-2b1495214ef5 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:43.151975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4ce70-cfac-4250-8368-a1544e1fed5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:43.153092 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4ce70-cfac-4250-8368-a1544e1fed5c admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:43.153463 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4ce70-cfac-4250-8368-a1544e1fed5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:43.153844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4ce70-cfac-4250-8368-a1544e1fed5c admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:43.154659 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4ce70-cfac-4250-8368-a1544e1fed5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:43.157006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4ce70-cfac-4250-8368-a1544e1fed5c admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:43.175522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 934/1843] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:43.178373 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf7d56-9f2a-4b7e-9a9d-3d7f117031d6 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:43.194442 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf7d56-9f2a-4b7e-9a9d-3d7f117031d6 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:43.200319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52d7ad-af09-417e-a789-d22d9a775a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:43.201147 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52d7ad-af09-417e-a789-d22d9a775a3d admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:43.201779 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52d7ad-af09-417e-a789-d22d9a775a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:43.202174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52d7ad-af09-417e-a789-d22d9a775a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:43.203003 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52d7ad-af09-417e-a789-d22d9a775a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:43.204964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52d7ad-af09-417e-a789-d22d9a775a3d admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:43.219745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 910/1844] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:43.219908 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4ce70-cfac-4250-8368-a1544e1fed5c admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:43.219987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d0a4ce70-cfac-4250-8368-a1544e1fed5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:43.221718 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52d7ad-af09-417e-a789-d22d9a775a3d admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:43.236659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52d7ad-af09-417e-a789-d22d9a775a3d admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:43.254411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4ce70-cfac-4250-8368-a1544e1fed5c admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:43.261499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83182940-c808-488d-9d47-46ec7ff4193b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:43.262488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83182940-c808-488d-9d47-46ec7ff4193b admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:43.263077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83182940-c808-488d-9d47-46ec7ff4193b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:43.263423 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83182940-c808-488d-9d47-46ec7ff4193b admin admin] PATH_INFO: `/v3/projects/a317fc7813824b0c94196bdf6e8ae81a/users/f859eae147354580875085d05f98e482/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:43.264044 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5fd23-0732-48e4-b1ec-3516d5f9f7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:43.266172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5fd23-0732-48e4-b1ec-3516d5f9f7a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:43.266466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5fd23-0732-48e4-b1ec-3516d5f9f7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:43.268278 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5fd23-0732-48e4-b1ec-3516d5f9f7a2 admin admin] PATH_INFO: `/v3/projects/6db74dbfceeb49d88746eee3cf4c8ed0` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:43.271163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5fd23-0732-48e4-b1ec-3516d5f9f7a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6db74dbfceeb49d88746eee3cf4c8ed0)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:43.282205 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5fd23-0732-48e4-b1ec-3516d5f9f7a2 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:43.301885 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83182940-c808-488d-9d47-46ec7ff4193b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a317fc7813824b0c94196bdf6e8ae81a, user_id=f859eae147354580875085d05f98e482, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:43.305806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83182940-c808-488d-9d47-46ec7ff4193b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:43.332909 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 911/1845] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:43 2022] PUT /identity/v3/projects/a317fc7813824b0c94196bdf6e8ae81a/users/f859eae147354580875085d05f98e482/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:43.334046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83182940-c808-488d-9d47-46ec7ff4193b admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:43.376798 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6ba5fd23-0732-48e4-b1ec-3516d5f9f7a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6db74dbfceeb49d88746eee3cf4c8ed0', 'request_id': 'req-6ba5fd23-0732-48e4-b1ec-3516d5f9f7a2'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:43.384264 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6ba5fd23-0732-48e4-b1ec-3516d5f9f7a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6db74dbfceeb49d88746eee3cf4c8ed0', 'request_id': 'req-6ba5fd23-0732-48e4-b1ec-3516d5f9f7a2'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:43.397395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6ba5fd23-0732-48e4-b1ec-3516d5f9f7a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6db74dbfceeb49d88746eee3cf4c8ed0', 'request_id': 'req-6ba5fd23-0732-48e4-b1ec-3516d5f9f7a2'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:43.415890 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6ba5fd23-0732-48e4-b1ec-3516d5f9f7a2 admin admin] The token cache is being invalidate because project 6db74dbfceeb49d88746eee3cf4c8ed0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:43.416416 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6ba5fd23-0732-48e4-b1ec-3516d5f9f7a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:43.417019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6ba5fd23-0732-48e4-b1ec-3516d5f9f7a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:43.423540 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 935/1846] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:43 2022] DELETE /identity/v3/projects/6db74dbfceeb49d88746eee3cf4c8ed0 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:45:43.426470 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0703e-fbec-4eb0-b386-3947f1e2877a None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:43.427405 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0703e-fbec-4eb0-b386-3947f1e2877a None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:43.427830 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0703e-fbec-4eb0-b386-3947f1e2877a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:43.430285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83182940-c808-488d-9d47-46ec7ff4193b admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:43.470091 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-e0d0703e-fbec-4eb0-b386-3947f1e2877a None None] MFA Rules not processed for user `f859eae147354580875085d05f98e482`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:43.472447 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e0d0703e-fbec-4eb0-b386-3947f1e2877a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:43.483041 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:43.520057 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916f3bc-e66f-4890-b2ed-2cb29229eafa tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de950003de87400f8f94a31e3ea23110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c30c5caee2f249a28b1eb4398c3e4d19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:43.520839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916f3bc-e66f-4890-b2ed-2cb29229eafa tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:43.521297 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916f3bc-e66f-4890-b2ed-2cb29229eafa tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:43.521555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916f3bc-e66f-4890-b2ed-2cb29229eafa tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:43.522752 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916f3bc-e66f-4890-b2ed-2cb29229eafa tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:43.522752 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916f3bc-e66f-4890-b2ed-2cb29229eafa tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:43.524432 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916f3bc-e66f-4890-b2ed-2cb29229eafa tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:43.542549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 912/1847] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:43.544370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacaa25-8e8d-40ff-a2be-b43d9d7e6671 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:43.544857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacaa25-8e8d-40ff-a2be-b43d9d7e6671 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:43.545312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacaa25-8e8d-40ff-a2be-b43d9d7e6671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:43.565910 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-fdacaa25-8e8d-40ff-a2be-b43d9d7e6671 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:43.567116 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-fdacaa25-8e8d-40ff-a2be-b43d9d7e6671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:43.575891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 936/1848] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:43.575891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d0703e-fbec-4eb0-b386-3947f1e2877a None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:43.575891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e0d0703e-fbec-4eb0-b386-3947f1e2877a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:43.614617 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:43.646018 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 913/1849] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:43.652317 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdacaa25-8e8d-40ff-a2be-b43d9d7e6671 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:43.652317 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-fdacaa25-8e8d-40ff-a2be-b43d9d7e6671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:43.664076 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdacaa25-8e8d-40ff-a2be-b43d9d7e6671 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:43.666488 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d0703e-fbec-4eb0-b386-3947f1e2877a None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:43.671689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdeae8-83ad-4df4-8de2-0f429088ff49 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de950003de87400f8f94a31e3ea23110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c30c5caee2f249a28b1eb4398c3e4d19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:43.671689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdeae8-83ad-4df4-8de2-0f429088ff49 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:43.671689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdeae8-83ad-4df4-8de2-0f429088ff49 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:43.671940 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdeae8-83ad-4df4-8de2-0f429088ff49 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:43.672511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e0f1a-ec5c-4643-840c-94caf30c434b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:43.673758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e0f1a-ec5c-4643-840c-94caf30c434b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:43.674087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e0f1a-ec5c-4643-840c-94caf30c434b admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:43.674494 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e0f1a-ec5c-4643-840c-94caf30c434b admin admin] PATH_INFO: `/v3/projects/c83bc3db4c6148e1b0cd19ad92312947` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:43.676184 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bdeae8-83ad-4df4-8de2-0f429088ff49 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:43.676364 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e0f1a-ec5c-4643-840c-94caf30c434b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c83bc3db4c6148e1b0cd19ad92312947)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:43.678698 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bdeae8-83ad-4df4-8de2-0f429088ff49 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:43.679857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e0f1a-ec5c-4643-840c-94caf30c434b admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:43.711391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 914/1850] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:43 2022] POST /identity/v3/projects => generated 353 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:43.711391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdeae8-83ad-4df4-8de2-0f429088ff49 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:43.713957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-13bdeae8-83ad-4df4-8de2-0f429088ff49 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:43.738460 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3a2e0f1a-ec5c-4643-840c-94caf30c434b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c83bc3db4c6148e1b0cd19ad92312947', 'request_id': 'req-3a2e0f1a-ec5c-4643-840c-94caf30c434b'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:43.746734 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3a2e0f1a-ec5c-4643-840c-94caf30c434b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c83bc3db4c6148e1b0cd19ad92312947', 'request_id': 'req-3a2e0f1a-ec5c-4643-840c-94caf30c434b'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:43.756342 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3a2e0f1a-ec5c-4643-840c-94caf30c434b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c83bc3db4c6148e1b0cd19ad92312947', 'request_id': 'req-3a2e0f1a-ec5c-4643-840c-94caf30c434b'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:43.768168 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3a2e0f1a-ec5c-4643-840c-94caf30c434b admin admin] The token cache is being invalidate because project c83bc3db4c6148e1b0cd19ad92312947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:43.768468 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3a2e0f1a-ec5c-4643-840c-94caf30c434b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:43.769293 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3a2e0f1a-ec5c-4643-840c-94caf30c434b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:43.770865 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 937/1851] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:43 2022] DELETE /identity/v3/projects/c83bc3db4c6148e1b0cd19ad92312947 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:43.772240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db653b5-2b61-4107-9fc9-e593915516ac None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:43.772525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db653b5-2b61-4107-9fc9-e593915516ac None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:43.772796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db653b5-2b61-4107-9fc9-e593915516ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:43.786575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdeae8-83ad-4df4-8de2-0f429088ff49 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:43.853059 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7a9a5-92a0-4520-b658-b4c67cc8b5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:43.853912 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7a9a5-92a0-4520-b658-b4c67cc8b5c7 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:43.854288 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7a9a5-92a0-4520-b658-b4c67cc8b5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:43.854665 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7a9a5-92a0-4520-b658-b4c67cc8b5c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:43.855502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7a9a5-92a0-4520-b658-b4c67cc8b5c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:43.857902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7a9a5-92a0-4520-b658-b4c67cc8b5c7 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:43.862516 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:43.894489 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 915/1852] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:43 2022] POST /identity/v3/projects => generated 367 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:43.894489 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7a9a5-92a0-4520-b658-b4c67cc8b5c7 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:43.896502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-40f7a9a5-92a0-4520-b658-b4c67cc8b5c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:43.925705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-3db653b5-2b61-4107-9fc9-e593915516ac None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:43.927254 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-3db653b5-2b61-4107-9fc9-e593915516ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:43.999988 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7a9a5-92a0-4520-b658-b4c67cc8b5c7 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:44.006589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8dcc5-a9a8-4df9-9be7-1041477e19a7 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de950003de87400f8f94a31e3ea23110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c30c5caee2f249a28b1eb4398c3e4d19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:44.007565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8dcc5-a9a8-4df9-9be7-1041477e19a7 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:44.007862 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8dcc5-a9a8-4df9-9be7-1041477e19a7 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:44.008149 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8dcc5-a9a8-4df9-9be7-1041477e19a7 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:44.008888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8dcc5-a9a8-4df9-9be7-1041477e19a7 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:44.009207 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8dcc5-a9a8-4df9-9be7-1041477e19a7 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:44.011105 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8dcc5-a9a8-4df9-9be7-1041477e19a7 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:44.021617 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 938/1853] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:44.021617 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db653b5-2b61-4107-9fc9-e593915516ac None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:44.024258 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-3db653b5-2b61-4107-9fc9-e593915516ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:44.034619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 916/1854] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:44.034619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8dcc5-a9a8-4df9-9be7-1041477e19a7 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:44.034619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-04e8dcc5-a9a8-4df9-9be7-1041477e19a7 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:44.038231 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db653b5-2b61-4107-9fc9-e593915516ac None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:44.043272 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb5238-c74b-4c77-a63a-cbde17099ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:44.044196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb5238-c74b-4c77-a63a-cbde17099ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:44.044563 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb5238-c74b-4c77-a63a-cbde17099ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:44.044758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb5238-c74b-4c77-a63a-cbde17099ac0 admin admin] PATH_INFO: `/v3/users/3fc2263237cd4a788f8461929c576892` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:44.046051 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb5238-c74b-4c77-a63a-cbde17099ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fc2263237cd4a788f8461929c576892)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:44.051878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8dcc5-a9a8-4df9-9be7-1041477e19a7 tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:44.055074 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb5238-c74b-4c77-a63a-cbde17099ac0 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:44.057561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e073f0c-fa8d-4ea9-addd-993ed0e76be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:44.058239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e073f0c-fa8d-4ea9-addd-993ed0e76be5 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:44.058522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e073f0c-fa8d-4ea9-addd-993ed0e76be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:44.058796 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e073f0c-fa8d-4ea9-addd-993ed0e76be5 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:44.059726 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e073f0c-fa8d-4ea9-addd-993ed0e76be5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:44.061690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e073f0c-fa8d-4ea9-addd-993ed0e76be5 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:44.085598 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 917/1855] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:44 2022] POST /identity/v3/users => generated 290 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:44.086570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e073f0c-fa8d-4ea9-addd-993ed0e76be5 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:44.097353 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e073f0c-fa8d-4ea9-addd-993ed0e76be5 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:44.103162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23634a99-151b-4136-b306-46f7ba6cb924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:44.104562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23634a99-151b-4136-b306-46f7ba6cb924 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:44.104847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23634a99-151b-4136-b306-46f7ba6cb924 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:44.105123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23634a99-151b-4136-b306-46f7ba6cb924 admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:44.106152 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23634a99-151b-4136-b306-46f7ba6cb924 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:44.106455 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23634a99-151b-4136-b306-46f7ba6cb924 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:44.108090 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23634a99-151b-4136-b306-46f7ba6cb924 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:44.120699 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-52bb5238-c74b-4c77-a63a-cbde17099ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fc2263237cd4a788f8461929c576892'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.123392 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 918/1856] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:44.123392 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23634a99-151b-4136-b306-46f7ba6cb924 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:44.133661 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23634a99-151b-4136-b306-46f7ba6cb924 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:44.134971 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:44.141682 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-52bb5238-c74b-4c77-a63a-cbde17099ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fc2263237cd4a788f8461929c576892'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.141874 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b17fe0-f06b-47aa-9489-1bc5fc67c5ba tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de950003de87400f8f94a31e3ea23110', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c30c5caee2f249a28b1eb4398c3e4d19', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:44.142888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b17fe0-f06b-47aa-9489-1bc5fc67c5ba tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:44.143174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b17fe0-f06b-47aa-9489-1bc5fc67c5ba tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:44.143498 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b17fe0-f06b-47aa-9489-1bc5fc67c5ba tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] PATH_INFO: `/v3/projects/ce19aa9f34844c998b6318ca1e52ee80` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:44.144838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b17fe0-f06b-47aa-9489-1bc5fc67c5ba tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ce19aa9f34844c998b6318ca1e52ee80)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:44.146778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b17fe0-f06b-47aa-9489-1bc5fc67c5ba tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:44.150413 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-52bb5238-c74b-4c77-a63a-cbde17099ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fc2263237cd4a788f8461929c576892'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.158406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-52bb5238-c74b-4c77-a63a-cbde17099ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fc2263237cd4a788f8461929c576892'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.159097 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-52bb5238-c74b-4c77-a63a-cbde17099ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fc2263237cd4a788f8461929c576892'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.165621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 939/1857] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:44 2022] DELETE /identity/v3/users/3fc2263237cd4a788f8461929c576892 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:44.166492 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb5238-c74b-4c77-a63a-cbde17099ac0 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:44.167952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-52bb5238-c74b-4c77-a63a-cbde17099ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:44.197700 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-f5b17fe0-f06b-47aa-9489-1bc5fc67c5ba tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce19aa9f34844c998b6318ca1e52ee80', 'request_id': 'req-f5b17fe0-f06b-47aa-9489-1bc5fc67c5ba'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.206466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-f5b17fe0-f06b-47aa-9489-1bc5fc67c5ba tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce19aa9f34844c998b6318ca1e52ee80', 'request_id': 'req-f5b17fe0-f06b-47aa-9489-1bc5fc67c5ba'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.213255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-f5b17fe0-f06b-47aa-9489-1bc5fc67c5ba tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce19aa9f34844c998b6318ca1e52ee80', 'request_id': 'req-f5b17fe0-f06b-47aa-9489-1bc5fc67c5ba'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.226803 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-f5b17fe0-f06b-47aa-9489-1bc5fc67c5ba tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] The token cache is being invalidate because project ce19aa9f34844c998b6318ca1e52ee80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:44.227188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-f5b17fe0-f06b-47aa-9489-1bc5fc67c5ba tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.227794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-f5b17fe0-f06b-47aa-9489-1bc5fc67c5ba tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.229623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 919/1858] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:44 2022] DELETE /identity/v3/projects/ce19aa9f34844c998b6318ca1e52ee80 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:44.233841 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b17fe0-f06b-47aa-9489-1bc5fc67c5ba tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:44.235464 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f5b17fe0-f06b-47aa-9489-1bc5fc67c5ba tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:44.246798 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb5238-c74b-4c77-a63a-cbde17099ac0 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:44.285162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:44.341522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e1766-99eb-47d8-9631-88e576d142c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:44.342241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e1766-99eb-47d8-9631-88e576d142c0 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:44.342241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e1766-99eb-47d8-9631-88e576d142c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:44.342825 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e1766-99eb-47d8-9631-88e576d142c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:44.343694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e1766-99eb-47d8-9631-88e576d142c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:44.345519 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e1766-99eb-47d8-9631-88e576d142c0 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:44.353635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b17fe0-f06b-47aa-9489-1bc5fc67c5ba tempest-VolumeQuotaClassesTest-1430582705 tempest-VolumeQuotaClassesTest-1430582705-project-admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:44.359333 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3fa8c-4604-4244-a958-f33dae4337e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:44.360107 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3fa8c-4604-4244-a958-f33dae4337e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:44.360420 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3fa8c-4604-4244-a958-f33dae4337e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:44.363812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 940/1859] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:44.363948 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3fa8c-4604-4244-a958-f33dae4337e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:44.363948 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3fa8c-4604-4244-a958-f33dae4337e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:44.364752 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3fa8c-4604-4244-a958-f33dae4337e3 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:44.365306 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e1766-99eb-47d8-9631-88e576d142c0 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:44.370240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-2b7e1766-99eb-47d8-9631-88e576d142c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:44.388100 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e1766-99eb-47d8-9631-88e576d142c0 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:44.396048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b41ff-4187-47d6-a693-718987178ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:44.396952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b41ff-4187-47d6-a693-718987178ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:44.397336 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b41ff-4187-47d6-a693-718987178ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:44.397520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b41ff-4187-47d6-a693-718987178ab7 admin admin] PATH_INFO: `/v3/projects/e785f4565e504d0caef18c405c8cfa6c/users/d0ac1142757242d5b0f240b671769639/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:44.407976 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 920/1860] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:44 2022] POST /identity/v3/projects => generated 383 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:44.412398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3fa8c-4604-4244-a958-f33dae4337e3 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:44.422192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b41ff-4187-47d6-a693-718987178ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e785f4565e504d0caef18c405c8cfa6c, user_id=d0ac1142757242d5b0f240b671769639, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:44.425353 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b41ff-4187-47d6-a693-718987178ab7 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:44.442590 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 941/1861] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:44 2022] PUT /identity/v3/projects/e785f4565e504d0caef18c405c8cfa6c/users/d0ac1142757242d5b0f240b671769639/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:44.448052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b41ff-4187-47d6-a693-718987178ab7 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:44.497672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3fa8c-4604-4244-a958-f33dae4337e3 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:44.528889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:44.547796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b41ff-4187-47d6-a693-718987178ab7 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:44.554220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf685dc-3d5c-469c-a1fb-f4e1a6ed473d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:44.556324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf685dc-3d5c-469c-a1fb-f4e1a6ed473d admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:44.556669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf685dc-3d5c-469c-a1fb-f4e1a6ed473d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:44.556948 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf685dc-3d5c-469c-a1fb-f4e1a6ed473d admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:44.559394 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf685dc-3d5c-469c-a1fb-f4e1a6ed473d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:44.561072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf685dc-3d5c-469c-a1fb-f4e1a6ed473d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:44.574316 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 942/1862] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:44.575273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf685dc-3d5c-469c-a1fb-f4e1a6ed473d admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:44.577027 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-8bf685dc-3d5c-469c-a1fb-f4e1a6ed473d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:44.588029 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79c825-5846-4679-a546-9d050d4be3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:44.588732 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79c825-5846-4679-a546-9d050d4be3cc admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:44.589026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79c825-5846-4679-a546-9d050d4be3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:44.589308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79c825-5846-4679-a546-9d050d4be3cc admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:44.590014 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79c825-5846-4679-a546-9d050d4be3cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:44.592166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79c825-5846-4679-a546-9d050d4be3cc admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:44.592748 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf685dc-3d5c-469c-a1fb-f4e1a6ed473d admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:44.598338 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce99754-d28b-4a00-8a9d-4a326a1f5223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:44.599247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce99754-d28b-4a00-8a9d-4a326a1f5223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:44.599571 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce99754-d28b-4a00-8a9d-4a326a1f5223 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:44.599848 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce99754-d28b-4a00-8a9d-4a326a1f5223 admin admin] PATH_INFO: `/v3/users/0b8e75d9eeef4a419af5d7f8d06234b5` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:44.601278 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce99754-d28b-4a00-8a9d-4a326a1f5223 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b8e75d9eeef4a419af5d7f8d06234b5)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:44.604484 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce99754-d28b-4a00-8a9d-4a326a1f5223 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:44.619020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 921/1863] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:44 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:45:44.620001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79c825-5846-4679-a546-9d050d4be3cc admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:44.621612 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-9a79c825-5846-4679-a546-9d050d4be3cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:44.635168 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79c825-5846-4679-a546-9d050d4be3cc admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:44.640493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df42b8a-3ab2-42c3-bb9a-ddfb83032554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:44.641899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df42b8a-3ab2-42c3-bb9a-ddfb83032554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:44.642453 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df42b8a-3ab2-42c3-bb9a-ddfb83032554 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:44.642716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df42b8a-3ab2-42c3-bb9a-ddfb83032554 admin admin] PATH_INFO: `/v3/projects/d3d6afb669de4bf8a4bec4405dac8199` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:44.643954 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df42b8a-3ab2-42c3-bb9a-ddfb83032554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3d6afb669de4bf8a4bec4405dac8199)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:44.646049 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df42b8a-3ab2-42c3-bb9a-ddfb83032554 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:44.665905 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-bce99754-d28b-4a00-8a9d-4a326a1f5223 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b8e75d9eeef4a419af5d7f8d06234b5'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.682642 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-bce99754-d28b-4a00-8a9d-4a326a1f5223 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b8e75d9eeef4a419af5d7f8d06234b5'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.692511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-bce99754-d28b-4a00-8a9d-4a326a1f5223 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b8e75d9eeef4a419af5d7f8d06234b5'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.700597 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:44.702589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-6df42b8a-3ab2-42c3-bb9a-ddfb83032554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3d6afb669de4bf8a4bec4405dac8199', 'request_id': 'req-6df42b8a-3ab2-42c3-bb9a-ddfb83032554'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.702986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-bce99754-d28b-4a00-8a9d-4a326a1f5223 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b8e75d9eeef4a419af5d7f8d06234b5'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.704449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-bce99754-d28b-4a00-8a9d-4a326a1f5223 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b8e75d9eeef4a419af5d7f8d06234b5'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.714168 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-6df42b8a-3ab2-42c3-bb9a-ddfb83032554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3d6afb669de4bf8a4bec4405dac8199', 'request_id': 'req-6df42b8a-3ab2-42c3-bb9a-ddfb83032554'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.719442 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 943/1864] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:44 2022] DELETE /identity/v3/users/0b8e75d9eeef4a419af5d7f8d06234b5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:44.720395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce99754-d28b-4a00-8a9d-4a326a1f5223 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:44.722175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-bce99754-d28b-4a00-8a9d-4a326a1f5223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:44.728527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-6df42b8a-3ab2-42c3-bb9a-ddfb83032554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3d6afb669de4bf8a4bec4405dac8199', 'request_id': 'req-6df42b8a-3ab2-42c3-bb9a-ddfb83032554'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.744072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-6df42b8a-3ab2-42c3-bb9a-ddfb83032554 admin admin] The token cache is being invalidate because project d3d6afb669de4bf8a4bec4405dac8199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:44.744363 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-6df42b8a-3ab2-42c3-bb9a-ddfb83032554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.745546 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-6df42b8a-3ab2-42c3-bb9a-ddfb83032554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:44.747994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 922/1865] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:44 2022] DELETE /identity/v3/projects/d3d6afb669de4bf8a4bec4405dac8199 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:44.748890 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df42b8a-3ab2-42c3-bb9a-ddfb83032554 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:44.750987 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-6df42b8a-3ab2-42c3-bb9a-ddfb83032554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:44.798437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce99754-d28b-4a00-8a9d-4a326a1f5223 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:44.805681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27c066-9f4c-4d2a-b1fa-a3934071e9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:44.806592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27c066-9f4c-4d2a-b1fa-a3934071e9b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:44.806873 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27c066-9f4c-4d2a-b1fa-a3934071e9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:44.807150 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27c066-9f4c-4d2a-b1fa-a3934071e9b5 admin admin] PATH_INFO: `/v3/projects/e785f4565e504d0caef18c405c8cfa6c/users/d0ac1142757242d5b0f240b671769639/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:44.809262 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb27c066-9f4c-4d2a-b1fa-a3934071e9b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e785f4565e504d0caef18c405c8cfa6c, user_id=d0ac1142757242d5b0f240b671769639, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:44.812080 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb27c066-9f4c-4d2a-b1fa-a3934071e9b5 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:44.824544 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 944/1866] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:44 2022] PUT /identity/v3/projects/e785f4565e504d0caef18c405c8cfa6c/users/d0ac1142757242d5b0f240b671769639/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:44.825908 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa11c8-2298-4187-bb25-8087e4820a9e None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:44.826201 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa11c8-2298-4187-bb25-8087e4820a9e None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:44.826585 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa11c8-2298-4187-bb25-8087e4820a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:44.836664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df42b8a-3ab2-42c3-bb9a-ddfb83032554 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:44.847692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d5aa11c8-2298-4187-bb25-8087e4820a9e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:44.849474 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d5aa11c8-2298-4187-bb25-8087e4820a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:44.926944 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5d5e6-0297-4791-8c4f-fbf0481f3981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:44.927650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5d5e6-0297-4791-8c4f-fbf0481f3981 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:44.928000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5d5e6-0297-4791-8c4f-fbf0481f3981 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:44.928207 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5d5e6-0297-4791-8c4f-fbf0481f3981 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:44.928942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5d5e6-0297-4791-8c4f-fbf0481f3981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:44.930621 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5d5e6-0297-4791-8c4f-fbf0481f3981 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:44.935243 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:44.942679 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 923/1867] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:44.943811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5d5e6-0297-4791-8c4f-fbf0481f3981 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:44.947376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a6c5d5e6-0297-4791-8c4f-fbf0481f3981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:44.950225 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 945/1868] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:44.951630 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e7143-9354-4896-b63c-4d5cb20f6b21 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:44.951987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e7143-9354-4896-b63c-4d5cb20f6b21 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:44.952338 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e7143-9354-4896-b63c-4d5cb20f6b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:44.962070 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5d5e6-0297-4791-8c4f-fbf0481f3981 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:44.966885 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555b3a5-d385-47f8-a5b3-becafcf7af47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:44.967799 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555b3a5-d385-47f8-a5b3-becafcf7af47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:44.968091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555b3a5-d385-47f8-a5b3-becafcf7af47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:44.968369 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555b3a5-d385-47f8-a5b3-becafcf7af47 admin admin] PATH_INFO: `/v3/users/9bf0226d2b914535817961f78191c8c4` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:44.969604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9555b3a5-d385-47f8-a5b3-becafcf7af47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bf0226d2b914535817961f78191c8c4)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:44.971595 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9555b3a5-d385-47f8-a5b3-becafcf7af47 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:44.981984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-023e7143-9354-4896-b63c-4d5cb20f6b21 None None] MFA Rules not processed for user `d0ac1142757242d5b0f240b671769639`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:44.983384 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-023e7143-9354-4896-b63c-4d5cb20f6b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:45.020050 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9555b3a5-d385-47f8-a5b3-becafcf7af47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bf0226d2b914535817961f78191c8c4'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:45.038641 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9555b3a5-d385-47f8-a5b3-becafcf7af47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bf0226d2b914535817961f78191c8c4'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:45.049135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9555b3a5-d385-47f8-a5b3-becafcf7af47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bf0226d2b914535817961f78191c8c4'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:45.049784 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9555b3a5-d385-47f8-a5b3-becafcf7af47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bf0226d2b914535817961f78191c8c4'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:45.052361 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:45.060820 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9555b3a5-d385-47f8-a5b3-becafcf7af47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bf0226d2b914535817961f78191c8c4'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:45.068133 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 946/1869] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:44 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:45.068817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e7143-9354-4896-b63c-4d5cb20f6b21 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:45.069272 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 924/1870] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:44 2022] DELETE /identity/v3/users/9bf0226d2b914535817961f78191c8c4 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:45.070120 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555b3a5-d385-47f8-a5b3-becafcf7af47 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:45.071775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-9555b3a5-d385-47f8-a5b3-becafcf7af47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:45.090038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e7143-9354-4896-b63c-4d5cb20f6b21 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:45.152466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555b3a5-d385-47f8-a5b3-becafcf7af47 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:45.161228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1073d-c8a7-4422-a6e0-748a7c5a15b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:45.161905 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1073d-c8a7-4422-a6e0-748a7c5a15b9 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:45.162187 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1073d-c8a7-4422-a6e0-748a7c5a15b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:45.162504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1073d-c8a7-4422-a6e0-748a7c5a15b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:45.166824 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-19f1073d-c8a7-4422-a6e0-748a7c5a15b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:45.174856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645efb2-8ca0-4439-903b-6a00c04b470a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:45.175573 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645efb2-8ca0-4439-903b-6a00c04b470a admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:45.175861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645efb2-8ca0-4439-903b-6a00c04b470a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:45.176141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645efb2-8ca0-4439-903b-6a00c04b470a admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:45.176907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645efb2-8ca0-4439-903b-6a00c04b470a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:45.178998 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f645efb2-8ca0-4439-903b-6a00c04b470a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:45.194170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1073d-c8a7-4422-a6e0-748a7c5a15b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:45.195559 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1073d-c8a7-4422-a6e0-748a7c5a15b9 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:45.196627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 947/1871] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:45.201773 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645efb2-8ca0-4439-903b-6a00c04b470a admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:45.204300 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f645efb2-8ca0-4439-903b-6a00c04b470a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:45.206800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 925/1872] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:45.207645 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1073d-c8a7-4422-a6e0-748a7c5a15b9 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:45.209051 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-19f1073d-c8a7-4422-a6e0-748a7c5a15b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:45.222666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1073d-c8a7-4422-a6e0-748a7c5a15b9 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:45.228012 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86c97d-dc86-4f14-bed6-45af3d173531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:45.229078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86c97d-dc86-4f14-bed6-45af3d173531 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:45.229361 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86c97d-dc86-4f14-bed6-45af3d173531 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:45.230137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86c97d-dc86-4f14-bed6-45af3d173531 admin admin] PATH_INFO: `/v3/projects/a8c6910b78a7491694428d35e3ef607b/users/2da7039a37aa43638981142a92dee5e3/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:45.248344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86c97d-dc86-4f14-bed6-45af3d173531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c6910b78a7491694428d35e3ef607b, user_id=2da7039a37aa43638981142a92dee5e3, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:45.251748 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86c97d-dc86-4f14-bed6-45af3d173531 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:45.266614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 926/1873] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:45 2022] PUT /identity/v3/projects/a8c6910b78a7491694428d35e3ef607b/users/2da7039a37aa43638981142a92dee5e3/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:45.281201 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d990e8-e86d-4175-8d6a-654e229f9fb4 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:45.281607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d990e8-e86d-4175-8d6a-654e229f9fb4 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:45.281946 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d990e8-e86d-4175-8d6a-654e229f9fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:45.289473 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f645efb2-8ca0-4439-903b-6a00c04b470a admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:45.315563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-a7d990e8-e86d-4175-8d6a-654e229f9fb4 None None] MFA Rules not processed for user `2da7039a37aa43638981142a92dee5e3`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:45.317563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a7d990e8-e86d-4175-8d6a-654e229f9fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:45.325250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:45.366989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17939e7b-28af-4f63-865c-2a6f54045a9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:45.368674 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17939e7b-28af-4f63-865c-2a6f54045a9b service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:45.369683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17939e7b-28af-4f63-865c-2a6f54045a9b service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:45.370288 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17939e7b-28af-4f63-865c-2a6f54045a9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:45.373624 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-17939e7b-28af-4f63-865c-2a6f54045a9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:45.408799 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 927/1874] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:45.413643 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d990e8-e86d-4175-8d6a-654e229f9fb4 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:45.468041 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17939e7b-28af-4f63-865c-2a6f54045a9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:45.469551 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17939e7b-28af-4f63-865c-2a6f54045a9b service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:45.482345 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 948/1875] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:45 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:45.515536 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d990e8-e86d-4175-8d6a-654e229f9fb4 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:45.526222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dae377-99c8-45d7-a032-d4dc359d995e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:45.527195 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dae377-99c8-45d7-a032-d4dc359d995e admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:45.528219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dae377-99c8-45d7-a032-d4dc359d995e admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:45.528219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dae377-99c8-45d7-a032-d4dc359d995e admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:45.528634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dae377-99c8-45d7-a032-d4dc359d995e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:45.530694 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dae377-99c8-45d7-a032-d4dc359d995e admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:45.536344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:45.547793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17939e7b-28af-4f63-865c-2a6f54045a9b service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:45.549775 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-17939e7b-28af-4f63-865c-2a6f54045a9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:45.560891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17939e7b-28af-4f63-865c-2a6f54045a9b service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:45.568439 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837547d-48eb-4400-9978-5e05c82c7f15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:45.569717 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837547d-48eb-4400-9978-5e05c82c7f15 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:45.570225 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837547d-48eb-4400-9978-5e05c82c7f15 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:45.570646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837547d-48eb-4400-9978-5e05c82c7f15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:45.579084 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b837547d-48eb-4400-9978-5e05c82c7f15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:45.583809 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 928/1876] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:45 2022] POST /identity/v3/projects => generated 381 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:45.584742 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dae377-99c8-45d7-a032-d4dc359d995e admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:45.668626 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837547d-48eb-4400-9978-5e05c82c7f15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:45.668876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dae377-99c8-45d7-a032-d4dc359d995e admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:45.670622 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837547d-48eb-4400-9978-5e05c82c7f15 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:45.674844 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417b630-7be4-4fad-8988-1144aab4ab59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:45.675675 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417b630-7be4-4fad-8988-1144aab4ab59 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:45.675962 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417b630-7be4-4fad-8988-1144aab4ab59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:45.676238 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417b630-7be4-4fad-8988-1144aab4ab59 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:45.677346 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417b630-7be4-4fad-8988-1144aab4ab59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:45.680142 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417b630-7be4-4fad-8988-1144aab4ab59 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:45.683938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 949/1877] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:45 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:45:45.711621 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 929/1878] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:45 2022] POST /identity/v3/users => generated 297 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:45.715666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837547d-48eb-4400-9978-5e05c82c7f15 service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:45.730270 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837547d-48eb-4400-9978-5e05c82c7f15 service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:45.737811 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e7451-d17e-4c43-9a7d-b386c470bbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:45.738591 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e7451-d17e-4c43-9a7d-b386c470bbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:45.739057 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e7451-d17e-4c43-9a7d-b386c470bbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:45.739467 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e7451-d17e-4c43-9a7d-b386c470bbe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:45.740383 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e7451-d17e-4c43-9a7d-b386c470bbe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:45.742383 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e7451-d17e-4c43-9a7d-b386c470bbe3 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:45.755836 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 950/1879] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:45.759678 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417b630-7be4-4fad-8988-1144aab4ab59 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:45.773609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417b630-7be4-4fad-8988-1144aab4ab59 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:45.789213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967292d4-f9e3-48d1-90ce-973e1a38b763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:45.790973 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967292d4-f9e3-48d1-90ce-973e1a38b763 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:45.791269 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967292d4-f9e3-48d1-90ce-973e1a38b763 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:45.791582 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967292d4-f9e3-48d1-90ce-973e1a38b763 admin admin] PATH_INFO: `/v3/projects/8ca292141909417fa92590a10ddabeaf/users/fd5ab70a24d543eba6335ab6276a6e95/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:45.828743 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967292d4-f9e3-48d1-90ce-973e1a38b763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ca292141909417fa92590a10ddabeaf, user_id=fd5ab70a24d543eba6335ab6276a6e95, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:45.833093 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967292d4-f9e3-48d1-90ce-973e1a38b763 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:45.855330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 930/1880] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:45 2022] PUT /identity/v3/projects/8ca292141909417fa92590a10ddabeaf/users/fd5ab70a24d543eba6335ab6276a6e95/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:45.859065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e7451-d17e-4c43-9a7d-b386c470bbe3 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:45.874279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e7451-d17e-4c43-9a7d-b386c470bbe3 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:45.985827 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7a3b5-e252-4574-9a33-ba7cd0e76263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:45.987064 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7a3b5-e252-4574-9a33-ba7cd0e76263 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:45.987577 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7a3b5-e252-4574-9a33-ba7cd0e76263 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:45.988177 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7a3b5-e252-4574-9a33-ba7cd0e76263 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:45.989128 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7a3b5-e252-4574-9a33-ba7cd0e76263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:45.992088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7a3b5-e252-4574-9a33-ba7cd0e76263 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:46.009705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 951/1881] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:46.011405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967292d4-f9e3-48d1-90ce-973e1a38b763 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:46.029250 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967292d4-f9e3-48d1-90ce-973e1a38b763 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:46.040592 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32802a87-61f5-4433-9eac-4eb27b222d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:46.044767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32802a87-61f5-4433-9eac-4eb27b222d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:46.045316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32802a87-61f5-4433-9eac-4eb27b222d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:46.045675 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32802a87-61f5-4433-9eac-4eb27b222d15 admin admin] PATH_INFO: `/v3/projects/8ca292141909417fa92590a10ddabeaf/users/fd5ab70a24d543eba6335ab6276a6e95/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:46.051166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32802a87-61f5-4433-9eac-4eb27b222d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ca292141909417fa92590a10ddabeaf, user_id=fd5ab70a24d543eba6335ab6276a6e95, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:46.057188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32802a87-61f5-4433-9eac-4eb27b222d15 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:46.087399 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 931/1882] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:46 2022] PUT /identity/v3/projects/8ca292141909417fa92590a10ddabeaf/users/fd5ab70a24d543eba6335ab6276a6e95/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:46.098687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88699e-8fb8-4876-ae16-ea6e1a99b627 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:46.099032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88699e-8fb8-4876-ae16-ea6e1a99b627 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:46.099897 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da88699e-8fb8-4876-ae16-ea6e1a99b627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:46.116502 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:46.136316 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-da88699e-8fb8-4876-ae16-ea6e1a99b627 None None] MFA Rules not processed for user `fd5ab70a24d543eba6335ab6276a6e95`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:46.137819 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-da88699e-8fb8-4876-ae16-ea6e1a99b627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:46.257687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 952/1883] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:46.272679 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32802a87-61f5-4433-9eac-4eb27b222d15 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:46.369424 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32802a87-61f5-4433-9eac-4eb27b222d15 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:46.375435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dd7a3-0a44-4f63-ac82-529702d471df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:46.375651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dd7a3-0a44-4f63-ac82-529702d471df service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:46.376043 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dd7a3-0a44-4f63-ac82-529702d471df service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:46.377304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dd7a3-0a44-4f63-ac82-529702d471df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:46.389419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dd7a3-0a44-4f63-ac82-529702d471df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:46.391709 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dd7a3-0a44-4f63-ac82-529702d471df service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:46.402666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 932/1884] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:46.412080 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88699e-8fb8-4876-ae16-ea6e1a99b627 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:46.413617 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-da88699e-8fb8-4876-ae16-ea6e1a99b627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:46.542337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dd7a3-0a44-4f63-ac82-529702d471df service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:46.548989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da88699e-8fb8-4876-ae16-ea6e1a99b627 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:46.559761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dd7a3-0a44-4f63-ac82-529702d471df service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:46.561481 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56fe76-c4ed-4123-8460-005aba6c64ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:46.562372 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56fe76-c4ed-4123-8460-005aba6c64ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:46.563655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56fe76-c4ed-4123-8460-005aba6c64ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:46.563655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56fe76-c4ed-4123-8460-005aba6c64ee admin admin] PATH_INFO: `/v3/projects/dccac589675d408c8fa056b4c088358b` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:46.564709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56fe76-c4ed-4123-8460-005aba6c64ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=dccac589675d408c8fa056b4c088358b)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:46.566636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa56fe76-c4ed-4123-8460-005aba6c64ee admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:46.572712 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0008689-362f-48f2-9e6c-d998ac902432 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:46.574429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0008689-362f-48f2-9e6c-d998ac902432 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:46.576258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0008689-362f-48f2-9e6c-d998ac902432 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:46.576889 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0008689-362f-48f2-9e6c-d998ac902432 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:46.661718 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-aa56fe76-c4ed-4123-8460-005aba6c64ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dccac589675d408c8fa056b4c088358b', 'request_id': 'req-aa56fe76-c4ed-4123-8460-005aba6c64ee'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:46.671713 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-aa56fe76-c4ed-4123-8460-005aba6c64ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dccac589675d408c8fa056b4c088358b', 'request_id': 'req-aa56fe76-c4ed-4123-8460-005aba6c64ee'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:46.691254 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-aa56fe76-c4ed-4123-8460-005aba6c64ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dccac589675d408c8fa056b4c088358b', 'request_id': 'req-aa56fe76-c4ed-4123-8460-005aba6c64ee'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:46.698136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:46.722430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0008689-362f-48f2-9e6c-d998ac902432 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:46.725504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0008689-362f-48f2-9e6c-d998ac902432 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:46.738898 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-aa56fe76-c4ed-4123-8460-005aba6c64ee admin admin] The token cache is being invalidate because project dccac589675d408c8fa056b4c088358b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:46.739502 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-aa56fe76-c4ed-4123-8460-005aba6c64ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:46.740076 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 933/1885] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:46.740633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-aa56fe76-c4ed-4123-8460-005aba6c64ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:46.741266 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0008689-362f-48f2-9e6c-d998ac902432 service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:46.747340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d0008689-362f-48f2-9e6c-d998ac902432 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:46.747738 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 953/1886] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:46 2022] DELETE /identity/v3/projects/dccac589675d408c8fa056b4c088358b => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:46.765583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0008689-362f-48f2-9e6c-d998ac902432 service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:46.775667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e4c00-687b-4e5f-afe6-562c4bdbaace None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:46.775966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e4c00-687b-4e5f-afe6-562c4bdbaace None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:46.776254 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e4c00-687b-4e5f-afe6-562c4bdbaace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:46.830553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-3a9e4c00-687b-4e5f-afe6-562c4bdbaace None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:46.831873 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-3a9e4c00-687b-4e5f-afe6-562c4bdbaace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:46.857499 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:46.858862 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386c3ff-29f6-454b-ab90-02c2824383b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:46.859850 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386c3ff-29f6-454b-ab90-02c2824383b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:46.860145 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386c3ff-29f6-454b-ab90-02c2824383b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:46.860440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386c3ff-29f6-454b-ab90-02c2824383b4 admin admin] PATH_INFO: `/v3/projects/ebd16beaed7747968b3afee4e9d0af19` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:46.861772 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386c3ff-29f6-454b-ab90-02c2824383b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebd16beaed7747968b3afee4e9d0af19)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:46.864472 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386c3ff-29f6-454b-ab90-02c2824383b4 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:46.931091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9386c3ff-29f6-454b-ab90-02c2824383b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebd16beaed7747968b3afee4e9d0af19', 'request_id': 'req-9386c3ff-29f6-454b-ab90-02c2824383b4'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:46.943958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 954/1887] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:46.948081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd829860-a83b-48db-be80-b085b81dea9d None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:46.948251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9386c3ff-29f6-454b-ab90-02c2824383b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebd16beaed7747968b3afee4e9d0af19', 'request_id': 'req-9386c3ff-29f6-454b-ab90-02c2824383b4'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:46.948432 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd829860-a83b-48db-be80-b085b81dea9d None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:46.948805 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd829860-a83b-48db-be80-b085b81dea9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:46.958917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9386c3ff-29f6-454b-ab90-02c2824383b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebd16beaed7747968b3afee4e9d0af19', 'request_id': 'req-9386c3ff-29f6-454b-ab90-02c2824383b4'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:46.969631 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-cd829860-a83b-48db-be80-b085b81dea9d None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:46.970870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-cd829860-a83b-48db-be80-b085b81dea9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:46.971964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9386c3ff-29f6-454b-ab90-02c2824383b4 admin admin] The token cache is being invalidate because project ebd16beaed7747968b3afee4e9d0af19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:46.972289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9386c3ff-29f6-454b-ab90-02c2824383b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:46.973842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9386c3ff-29f6-454b-ab90-02c2824383b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:46.976536 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 934/1888] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:46 2022] DELETE /identity/v3/projects/ebd16beaed7747968b3afee4e9d0af19 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:46.981200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386c3ff-29f6-454b-ab90-02c2824383b4 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:46.985319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-9386c3ff-29f6-454b-ab90-02c2824383b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:47.058533 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 955/1889] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:45:47.062671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd829860-a83b-48db-be80-b085b81dea9d None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:47.081674 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd829860-a83b-48db-be80-b085b81dea9d None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:47.090871 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f49e0-e6d3-440c-bb51-16e490c7a9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:47.090871 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f49e0-e6d3-440c-bb51-16e490c7a9cf admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:47.090871 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f49e0-e6d3-440c-bb51-16e490c7a9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:47.090871 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f49e0-e6d3-440c-bb51-16e490c7a9cf admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:47.092856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386c3ff-29f6-454b-ab90-02c2824383b4 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:47.095643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f49e0-e6d3-440c-bb51-16e490c7a9cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:47.096225 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f49e0-e6d3-440c-bb51-16e490c7a9cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:47.098170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc9da2-3e34-4dbd-aeba-35de0719126c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:47.099128 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f49e0-e6d3-440c-bb51-16e490c7a9cf admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:47.099388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc9da2-3e34-4dbd-aeba-35de0719126c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:47.099794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc9da2-3e34-4dbd-aeba-35de0719126c admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:47.100167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfc9da2-3e34-4dbd-aeba-35de0719126c admin admin] PATH_INFO: `/v3/users/de950003de87400f8f94a31e3ea23110` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:47.101747 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc9da2-3e34-4dbd-aeba-35de0719126c admin admin] RBAC: Authorizing `identity:delete_user(user_id=de950003de87400f8f94a31e3ea23110)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:47.104003 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfc9da2-3e34-4dbd-aeba-35de0719126c admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:47.118079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:47.120911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 956/1890] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:47.121937 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f49e0-e6d3-440c-bb51-16e490c7a9cf admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:47.138032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f49e0-e6d3-440c-bb51-16e490c7a9cf admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:47.143308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92dce4-bd52-4e5d-ac0e-537534ad7e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:47.144079 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92dce4-bd52-4e5d-ac0e-537534ad7e4e admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:47.144334 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92dce4-bd52-4e5d-ac0e-537534ad7e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:47.144608 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92dce4-bd52-4e5d-ac0e-537534ad7e4e admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:47.145328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92dce4-bd52-4e5d-ac0e-537534ad7e4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:47.147224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92dce4-bd52-4e5d-ac0e-537534ad7e4e admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:47.164033 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8cfc9da2-3e34-4dbd-aeba-35de0719126c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de950003de87400f8f94a31e3ea23110'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:47.184888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 957/1891] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:47 2022] POST /identity/v3/projects => generated 399 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:47.184888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92dce4-bd52-4e5d-ac0e-537534ad7e4e admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:47.185080 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8cfc9da2-3e34-4dbd-aeba-35de0719126c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de950003de87400f8f94a31e3ea23110'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:47.186731 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ae92dce4-bd52-4e5d-ac0e-537534ad7e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:47.197642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8cfc9da2-3e34-4dbd-aeba-35de0719126c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de950003de87400f8f94a31e3ea23110'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:47.198295 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8cfc9da2-3e34-4dbd-aeba-35de0719126c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de950003de87400f8f94a31e3ea23110'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:47.209147 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8cfc9da2-3e34-4dbd-aeba-35de0719126c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de950003de87400f8f94a31e3ea23110'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:47.218549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 935/1892] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:46 2022] DELETE /identity/v3/users/de950003de87400f8f94a31e3ea23110 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:47.220826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc9da2-3e34-4dbd-aeba-35de0719126c admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:47.223642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-8cfc9da2-3e34-4dbd-aeba-35de0719126c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:47.323876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfc9da2-3e34-4dbd-aeba-35de0719126c admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:47.330949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ad9aa-ee42-416a-ab56-63cf0bf91de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:47.331676 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ad9aa-ee42-416a-ab56-63cf0bf91de8 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:47.331958 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ad9aa-ee42-416a-ab56-63cf0bf91de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:47.332644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ad9aa-ee42-416a-ab56-63cf0bf91de8 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:47.333248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ad9aa-ee42-416a-ab56-63cf0bf91de8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:47.336185 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ad9aa-ee42-416a-ab56-63cf0bf91de8 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:47.338856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92dce4-bd52-4e5d-ac0e-537534ad7e4e admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:47.366573 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1108675-2230-4746-a329-4df07b5b9ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:47.367544 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1108675-2230-4746-a329-4df07b5b9ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:47.367835 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1108675-2230-4746-a329-4df07b5b9ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:47.368115 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1108675-2230-4746-a329-4df07b5b9ef4 admin admin] PATH_INFO: `/v3/users/2da7039a37aa43638981142a92dee5e3` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:47.368381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 936/1893] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:47 2022] POST /identity/v3/users => generated 300 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:47.369500 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1108675-2230-4746-a329-4df07b5b9ef4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2da7039a37aa43638981142a92dee5e3)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:47.371541 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1108675-2230-4746-a329-4df07b5b9ef4 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:47.372669 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ad9aa-ee42-416a-ab56-63cf0bf91de8 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:47.391064 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ad9aa-ee42-416a-ab56-63cf0bf91de8 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:47.403389 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13284e5-28c0-4bae-a842-cd67384ed11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:47.407527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13284e5-28c0-4bae-a842-cd67384ed11b admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:47.407817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13284e5-28c0-4bae-a842-cd67384ed11b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:47.408125 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13284e5-28c0-4bae-a842-cd67384ed11b admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:47.409216 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13284e5-28c0-4bae-a842-cd67384ed11b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:47.416473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13284e5-28c0-4bae-a842-cd67384ed11b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:47.430697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:47.435860 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 937/1894] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:47.437302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13284e5-28c0-4bae-a842-cd67384ed11b admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:47.447986 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13284e5-28c0-4bae-a842-cd67384ed11b admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:47.451819 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-a1108675-2230-4746-a329-4df07b5b9ef4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2da7039a37aa43638981142a92dee5e3'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:47.457107 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29a0f6-365f-43b4-a0b9-94a72fab668d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:47.457810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29a0f6-365f-43b4-a0b9-94a72fab668d admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:47.458126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29a0f6-365f-43b4-a0b9-94a72fab668d admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:47.458456 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29a0f6-365f-43b4-a0b9-94a72fab668d admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:47.459224 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29a0f6-365f-43b4-a0b9-94a72fab668d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:47.460942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29a0f6-365f-43b4-a0b9-94a72fab668d admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:47.473477 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 938/1895] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:45:47.478803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-a1108675-2230-4746-a329-4df07b5b9ef4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2da7039a37aa43638981142a92dee5e3'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:47.479383 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29a0f6-365f-43b4-a0b9-94a72fab668d admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:47.490851 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-a1108675-2230-4746-a329-4df07b5b9ef4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2da7039a37aa43638981142a92dee5e3'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:47.494101 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29a0f6-365f-43b4-a0b9-94a72fab668d admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:47.499825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938d2e2-425d-4e75-a83c-f92df4f2b06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:47.500805 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-a1108675-2230-4746-a329-4df07b5b9ef4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2da7039a37aa43638981142a92dee5e3'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:47.501236 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938d2e2-425d-4e75-a83c-f92df4f2b06a admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:47.501548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938d2e2-425d-4e75-a83c-f92df4f2b06a admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:47.501804 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938d2e2-425d-4e75-a83c-f92df4f2b06a admin admin] PATH_INFO: `/v3/projects/eeddcf9cf4094ed78967714498b6eb4a/users/4b4d92c9354e429b876a006ce4d0307f/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:47.502463 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-a1108675-2230-4746-a329-4df07b5b9ef4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2da7039a37aa43638981142a92dee5e3'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:47.510147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 958/1896] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:47 2022] DELETE /identity/v3/users/2da7039a37aa43638981142a92dee5e3 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:47.514472 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755856dd-c603-428a-83fd-6705bbc0ff1f None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:47.514752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755856dd-c603-428a-83fd-6705bbc0ff1f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:47.515744 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755856dd-c603-428a-83fd-6705bbc0ff1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:47.549924 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938d2e2-425d-4e75-a83c-f92df4f2b06a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeddcf9cf4094ed78967714498b6eb4a, user_id=4b4d92c9354e429b876a006ce4d0307f, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:47.554531 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938d2e2-425d-4e75-a83c-f92df4f2b06a admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:47.581656 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-755856dd-c603-428a-83fd-6705bbc0ff1f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:47.582890 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-755856dd-c603-428a-83fd-6705bbc0ff1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:47.625919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 939/1897] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:47 2022] PUT /identity/v3/projects/eeddcf9cf4094ed78967714498b6eb4a/users/4b4d92c9354e429b876a006ce4d0307f/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:47.644353 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1558f08-1ead-4e6d-a2f1-c82df4c83a5d None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:47.645734 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1558f08-1ead-4e6d-a2f1-c82df4c83a5d None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:47.645734 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1558f08-1ead-4e6d-a2f1-c82df4c83a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:47.689011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c1558f08-1ead-4e6d-a2f1-c82df4c83a5d None None] MFA Rules not processed for user `4b4d92c9354e429b876a006ce4d0307f`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:47.691582 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c1558f08-1ead-4e6d-a2f1-c82df4c83a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:47.705165 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 959/1898] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:45:47.718654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755856dd-c603-428a-83fd-6705bbc0ff1f None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:47.720641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-755856dd-c603-428a-83fd-6705bbc0ff1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:47.740434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:DELETE=10,keystone:SELECT=85,keystone:INSERT=5 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:47.823863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 940/1899] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:47 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:47.824653 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1558f08-1ead-4e6d-a2f1-c82df4c83a5d None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:47.826078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c1558f08-1ead-4e6d-a2f1-c82df4c83a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:47.842072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1558f08-1ead-4e6d-a2f1-c82df4c83a5d None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:47.849605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929a4f1-f585-45e4-b6cb-be276ccf90d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:47.850888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929a4f1-f585-45e4-b6cb-be276ccf90d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:47.851591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929a4f1-f585-45e4-b6cb-be276ccf90d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:47.852911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929a4f1-f585-45e4-b6cb-be276ccf90d0 admin admin] PATH_INFO: `/v3/projects/c30c5caee2f249a28b1eb4398c3e4d19` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:47.854386 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929a4f1-f585-45e4-b6cb-be276ccf90d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c30c5caee2f249a28b1eb4398c3e4d19)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:47.857739 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929a4f1-f585-45e4-b6cb-be276ccf90d0 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:47.887691 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755856dd-c603-428a-83fd-6705bbc0ff1f None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:47.897759 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf1c83-7fbc-42b9-b705-bf9e4646f367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:47.898324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf1c83-7fbc-42b9-b705-bf9e4646f367 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:47.898612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf1c83-7fbc-42b9-b705-bf9e4646f367 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:47.898988 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf1c83-7fbc-42b9-b705-bf9e4646f367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:47.931227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf1c83-7fbc-42b9-b705-bf9e4646f367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:47.931227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf1c83-7fbc-42b9-b705-bf9e4646f367 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:47.939692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 960/1900] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:47.940649 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf1c83-7fbc-42b9-b705-bf9e4646f367 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:47.942094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ddbf1c83-7fbc-42b9-b705-bf9e4646f367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:47.960372 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf1c83-7fbc-42b9-b705-bf9e4646f367 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:47.970337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-6929a4f1-f585-45e4-b6cb-be276ccf90d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c30c5caee2f249a28b1eb4398c3e4d19', 'request_id': 'req-6929a4f1-f585-45e4-b6cb-be276ccf90d0'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:47.980060 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-6929a4f1-f585-45e4-b6cb-be276ccf90d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c30c5caee2f249a28b1eb4398c3e4d19', 'request_id': 'req-6929a4f1-f585-45e4-b6cb-be276ccf90d0'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:47.987779 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-6929a4f1-f585-45e4-b6cb-be276ccf90d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c30c5caee2f249a28b1eb4398c3e4d19', 'request_id': 'req-6929a4f1-f585-45e4-b6cb-be276ccf90d0'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:47.999743 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-6929a4f1-f585-45e4-b6cb-be276ccf90d0 admin admin] The token cache is being invalidate because project c30c5caee2f249a28b1eb4398c3e4d19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:48.000077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-6929a4f1-f585-45e4-b6cb-be276ccf90d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:48.001631 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-6929a4f1-f585-45e4-b6cb-be276ccf90d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:48.006664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 941/1901] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:47 2022] DELETE /identity/v3/projects/c30c5caee2f249a28b1eb4398c3e4d19 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:48.008380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cc4e7-2f35-4f00-9441-fe4352a9ea79 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:48.008730 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cc4e7-2f35-4f00-9441-fe4352a9ea79 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:48.009072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cc4e7-2f35-4f00-9441-fe4352a9ea79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:48.025627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:48.030909 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-ef1cc4e7-2f35-4f00-9441-fe4352a9ea79 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:48.032258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ef1cc4e7-2f35-4f00-9441-fe4352a9ea79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:48.043244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2bf1f-4311-47ed-8790-b6366344f3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:48.044754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2bf1f-4311-47ed-8790-b6366344f3e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:48.045198 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2bf1f-4311-47ed-8790-b6366344f3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:48.045642 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2bf1f-4311-47ed-8790-b6366344f3e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:48.046413 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d2bf1f-4311-47ed-8790-b6366344f3e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:48.048412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d2bf1f-4311-47ed-8790-b6366344f3e3 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:48.079400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 961/1902] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:47 2022] POST /identity/v3/projects => generated 401 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:48.083471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2bf1f-4311-47ed-8790-b6366344f3e3 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:48.085436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-78d2bf1f-4311-47ed-8790-b6366344f3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:48.107336 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 942/1903] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:48.116575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2d12a-3d0d-41ab-ad0b-162db5dc0d66 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:48.117130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2d12a-3d0d-41ab-ad0b-162db5dc0d66 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:48.117414 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2d12a-3d0d-41ab-ad0b-162db5dc0d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:48.140119 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-27f2d12a-3d0d-41ab-ad0b-162db5dc0d66 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:48.142017 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-27f2d12a-3d0d-41ab-ad0b-162db5dc0d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:48.182441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2bf1f-4311-47ed-8790-b6366344f3e3 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:48.191429 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969986cd-a862-4b0c-afe1-a571b20d63f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:48.191429 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969986cd-a862-4b0c-afe1-a571b20d63f7 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:48.191429 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969986cd-a862-4b0c-afe1-a571b20d63f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:48.191429 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969986cd-a862-4b0c-afe1-a571b20d63f7 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:48.192137 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969986cd-a862-4b0c-afe1-a571b20d63f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:48.194810 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969986cd-a862-4b0c-afe1-a571b20d63f7 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:48.208453 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:48.224961 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 962/1904] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:48 2022] POST /identity/v3/users => generated 307 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:45:48.228856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969986cd-a862-4b0c-afe1-a571b20d63f7 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:48.233280 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 943/1905] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:45:48.237878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2d12a-3d0d-41ab-ad0b-162db5dc0d66 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:48.242453 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969986cd-a862-4b0c-afe1-a571b20d63f7 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:48.247533 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd37afd-f0ff-44ff-ad16-8f12f9f46cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:48.248217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd37afd-f0ff-44ff-ad16-8f12f9f46cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:48.248647 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd37afd-f0ff-44ff-ad16-8f12f9f46cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:48.249098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd37afd-f0ff-44ff-ad16-8f12f9f46cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:48.249840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd37afd-f0ff-44ff-ad16-8f12f9f46cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:48.252197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd37afd-f0ff-44ff-ad16-8f12f9f46cb2 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:48.252998 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2d12a-3d0d-41ab-ad0b-162db5dc0d66 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:48.258277 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e322b08-0575-4469-ba28-75a512ea2bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:48.259071 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e322b08-0575-4469-ba28-75a512ea2bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:48.259389 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e322b08-0575-4469-ba28-75a512ea2bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:48.260079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e322b08-0575-4469-ba28-75a512ea2bc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:48.260780 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e322b08-0575-4469-ba28-75a512ea2bc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:48.261091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e322b08-0575-4469-ba28-75a512ea2bc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:48.262729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e322b08-0575-4469-ba28-75a512ea2bc6 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:48.266741 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 963/1906] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:48.270793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd37afd-f0ff-44ff-ad16-8f12f9f46cb2 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:48.277376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 944/1907] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:48.281971 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e322b08-0575-4469-ba28-75a512ea2bc6 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:48.283468 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd37afd-f0ff-44ff-ad16-8f12f9f46cb2 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:48.288789 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f595cb2-a56c-42e6-a7b3-c47897eda267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:48.289712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f595cb2-a56c-42e6-a7b3-c47897eda267 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:48.289997 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f595cb2-a56c-42e6-a7b3-c47897eda267 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:48.290286 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f595cb2-a56c-42e6-a7b3-c47897eda267 admin admin] PATH_INFO: `/v3/projects/2aae088dfa144636bcfa79d0643a6f19/users/590c10c9351748deac98608e04380b4c/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:48.292946 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e322b08-0575-4469-ba28-75a512ea2bc6 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:48.300537 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7158500-daeb-4189-abf2-e7df80b33f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:48.301435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7158500-daeb-4189-abf2-e7df80b33f45 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:48.301705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7158500-daeb-4189-abf2-e7df80b33f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:48.302042 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7158500-daeb-4189-abf2-e7df80b33f45 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:48.302958 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7158500-daeb-4189-abf2-e7df80b33f45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:48.305347 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7158500-daeb-4189-abf2-e7df80b33f45 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:48.306632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f595cb2-a56c-42e6-a7b3-c47897eda267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aae088dfa144636bcfa79d0643a6f19, user_id=590c10c9351748deac98608e04380b4c, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:48.309327 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f595cb2-a56c-42e6-a7b3-c47897eda267 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:48.326747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 964/1908] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:48 2022] PUT /identity/v3/projects/2aae088dfa144636bcfa79d0643a6f19/users/590c10c9351748deac98608e04380b4c/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:48.330813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f595cb2-a56c-42e6-a7b3-c47897eda267 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:48.339789 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 945/1909] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:48 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:45:48.344582 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7158500-daeb-4189-abf2-e7df80b33f45 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:48.418574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f595cb2-a56c-42e6-a7b3-c47897eda267 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:48.437273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7158500-daeb-4189-abf2-e7df80b33f45 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:48.442590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ee8e5-5cce-4d0d-b557-ec6816b74cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:48.443385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ee8e5-5cce-4d0d-b557-ec6816b74cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:48.443742 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ee8e5-5cce-4d0d-b557-ec6816b74cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:48.444080 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ee8e5-5cce-4d0d-b557-ec6816b74cf4 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:48.444850 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ee8e5-5cce-4d0d-b557-ec6816b74cf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:48.447417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ee8e5-5cce-4d0d-b557-ec6816b74cf4 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:48.460379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:48.470962 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 946/1910] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:48 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:45:48.472124 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ee8e5-5cce-4d0d-b557-ec6816b74cf4 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:48.475499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-866ee8e5-5cce-4d0d-b557-ec6816b74cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:48.489080 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ee8e5-5cce-4d0d-b557-ec6816b74cf4 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:48.492476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2945a7c-86c3-4c06-afc2-3d94c7eda179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:48.493159 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2945a7c-86c3-4c06-afc2-3d94c7eda179 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:48.493441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2945a7c-86c3-4c06-afc2-3d94c7eda179 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:48.493718 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2945a7c-86c3-4c06-afc2-3d94c7eda179 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:48.494490 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bcabb-e455-4778-a6c3-c0f0e86cb5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:48.494628 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2945a7c-86c3-4c06-afc2-3d94c7eda179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:48.495511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bcabb-e455-4778-a6c3-c0f0e86cb5a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:48.496257 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2945a7c-86c3-4c06-afc2-3d94c7eda179 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:48.496537 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bcabb-e455-4778-a6c3-c0f0e86cb5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:48.496886 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bcabb-e455-4778-a6c3-c0f0e86cb5a9 admin admin] PATH_INFO: `/v3/projects/a8c6910b78a7491694428d35e3ef607b` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:48.498264 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bcabb-e455-4778-a6c3-c0f0e86cb5a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8c6910b78a7491694428d35e3ef607b)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:48.501078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bcabb-e455-4778-a6c3-c0f0e86cb5a9 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:48.508525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 965/1911] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:45:48.509829 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2945a7c-86c3-4c06-afc2-3d94c7eda179 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:48.524127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2945a7c-86c3-4c06-afc2-3d94c7eda179 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:48.540308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04605dc-254d-4956-ab94-6a76c5cd3472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:48.541225 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04605dc-254d-4956-ab94-6a76c5cd3472 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:48.541534 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04605dc-254d-4956-ab94-6a76c5cd3472 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:48.541816 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04605dc-254d-4956-ab94-6a76c5cd3472 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:48.542578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04605dc-254d-4956-ab94-6a76c5cd3472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:48.544264 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04605dc-254d-4956-ab94-6a76c5cd3472 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:48.555769 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 966/1912] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:48.556608 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04605dc-254d-4956-ab94-6a76c5cd3472 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:48.559392 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-540bcabb-e455-4778-a6c3-c0f0e86cb5a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8c6910b78a7491694428d35e3ef607b', 'request_id': 'req-540bcabb-e455-4778-a6c3-c0f0e86cb5a9'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:48.567702 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04605dc-254d-4956-ab94-6a76c5cd3472 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:48.569362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-540bcabb-e455-4778-a6c3-c0f0e86cb5a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8c6910b78a7491694428d35e3ef607b', 'request_id': 'req-540bcabb-e455-4778-a6c3-c0f0e86cb5a9'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:48.576524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-540bcabb-e455-4778-a6c3-c0f0e86cb5a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8c6910b78a7491694428d35e3ef607b', 'request_id': 'req-540bcabb-e455-4778-a6c3-c0f0e86cb5a9'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:48.589903 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-540bcabb-e455-4778-a6c3-c0f0e86cb5a9 admin admin] The token cache is being invalidate because project a8c6910b78a7491694428d35e3ef607b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:48.590381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-540bcabb-e455-4778-a6c3-c0f0e86cb5a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:48.591210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-540bcabb-e455-4778-a6c3-c0f0e86cb5a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:48.592841 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 947/1913] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:48 2022] DELETE /identity/v3/projects/a8c6910b78a7491694428d35e3ef607b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:48.596659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bcabb-e455-4778-a6c3-c0f0e86cb5a9 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:48.598239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-540bcabb-e455-4778-a6c3-c0f0e86cb5a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:48.626087 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:48.633154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c3ec2-44d6-43f7-9540-475de46a913a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:48.634062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c3ec2-44d6-43f7-9540-475de46a913a admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:48.634349 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c3ec2-44d6-43f7-9540-475de46a913a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:48.634851 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c3ec2-44d6-43f7-9540-475de46a913a admin admin] PATH_INFO: `/v3/projects/2aae088dfa144636bcfa79d0643a6f19/users/590c10c9351748deac98608e04380b4c/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:48.637123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c3ec2-44d6-43f7-9540-475de46a913a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aae088dfa144636bcfa79d0643a6f19, user_id=590c10c9351748deac98608e04380b4c, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:48.640016 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c3ec2-44d6-43f7-9540-475de46a913a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:48.669399 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 967/1914] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:48 2022] PUT /identity/v3/projects/2aae088dfa144636bcfa79d0643a6f19/users/590c10c9351748deac98608e04380b4c/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:48.669399 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c3ec2-44d6-43f7-9540-475de46a913a admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:48.672464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5a4c3ec2-44d6-43f7-9540-475de46a913a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:48.690634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bcabb-e455-4778-a6c3-c0f0e86cb5a9 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:48.794053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042682e-a74a-464a-9ccf-56533e8a8a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:48.795610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042682e-a74a-464a-9ccf-56533e8a8a30 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:48.795947 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042682e-a74a-464a-9ccf-56533e8a8a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:48.796224 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042682e-a74a-464a-9ccf-56533e8a8a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:48.797802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042682e-a74a-464a-9ccf-56533e8a8a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:48.799493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042682e-a74a-464a-9ccf-56533e8a8a30 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:48.805876 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c3ec2-44d6-43f7-9540-475de46a913a admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:48.813535 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 948/1915] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:48.818075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0a6e1-a3f3-4d71-bf3d-5c7b83b637bd None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:48.818341 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0a6e1-a3f3-4d71-bf3d-5c7b83b637bd None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:48.818726 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0a6e1-a3f3-4d71-bf3d-5c7b83b637bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:48.821806 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ea3d9-b398-410c-b64f-66fcddaa17f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:48.822983 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ea3d9-b398-410c-b64f-66fcddaa17f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:48.824255 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ea3d9-b398-410c-b64f-66fcddaa17f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:48.824587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ea3d9-b398-410c-b64f-66fcddaa17f5 admin admin] PATH_INFO: `/v3/users/fd5ab70a24d543eba6335ab6276a6e95` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:48.826289 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ea3d9-b398-410c-b64f-66fcddaa17f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd5ab70a24d543eba6335ab6276a6e95)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:48.828830 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ea3d9-b398-410c-b64f-66fcddaa17f5 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:48.868815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-44a0a6e1-a3f3-4d71-bf3d-5c7b83b637bd None None] MFA Rules not processed for user `590c10c9351748deac98608e04380b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:48.872170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-44a0a6e1-a3f3-4d71-bf3d-5c7b83b637bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:48.944098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-429ea3d9-b398-410c-b64f-66fcddaa17f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd5ab70a24d543eba6335ab6276a6e95'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:48.949074 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:48.964132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-429ea3d9-b398-410c-b64f-66fcddaa17f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd5ab70a24d543eba6335ab6276a6e95'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:48.977501 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-429ea3d9-b398-410c-b64f-66fcddaa17f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd5ab70a24d543eba6335ab6276a6e95'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:48.978920 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 949/1916] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:48 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:48.980546 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0a6e1-a3f3-4d71-bf3d-5c7b83b637bd None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:48.994864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0a6e1-a3f3-4d71-bf3d-5c7b83b637bd None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:48.996394 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-429ea3d9-b398-410c-b64f-66fcddaa17f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd5ab70a24d543eba6335ab6276a6e95'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:48.998536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-429ea3d9-b398-410c-b64f-66fcddaa17f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd5ab70a24d543eba6335ab6276a6e95'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:48.999644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47f894-2b7d-4a8f-8856-8b9560fdd101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:49.000591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47f894-2b7d-4a8f-8856-8b9560fdd101 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:49.000898 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47f894-2b7d-4a8f-8856-8b9560fdd101 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:49.001176 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47f894-2b7d-4a8f-8856-8b9560fdd101 admin admin] PATH_INFO: `/v3/projects/a642d4e26a324e8fa0bdd46543b04b26/users/035c6bb2d90e48c48a04a0ef095d511c/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:49.006825 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 968/1917] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:48 2022] DELETE /identity/v3/users/fd5ab70a24d543eba6335ab6276a6e95 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:49.007697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ea3d9-b398-410c-b64f-66fcddaa17f5 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:49.009046 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-429ea3d9-b398-410c-b64f-66fcddaa17f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:49.022919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47f894-2b7d-4a8f-8856-8b9560fdd101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a642d4e26a324e8fa0bdd46543b04b26, user_id=035c6bb2d90e48c48a04a0ef095d511c, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:49.024233 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47f894-2b7d-4a8f-8856-8b9560fdd101 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:49.040385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 950/1918] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:48 2022] PUT /identity/v3/projects/a642d4e26a324e8fa0bdd46543b04b26/users/035c6bb2d90e48c48a04a0ef095d511c/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:49.051683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624350c-b9d1-4e91-8583-63714e281336 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:49.051979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624350c-b9d1-4e91-8583-63714e281336 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:49.052508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624350c-b9d1-4e91-8583-63714e281336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:49.117722 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ea3d9-b398-410c-b64f-66fcddaa17f5 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:49.121126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-5624350c-b9d1-4e91-8583-63714e281336 None None] MFA Rules not processed for user `035c6bb2d90e48c48a04a0ef095d511c`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:49.128275 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-5624350c-b9d1-4e91-8583-63714e281336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:49.164983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:49.213897 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664dd73-5481-436c-b2b5-84887c71940a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:49.215739 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664dd73-5481-436c-b2b5-84887c71940a service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:49.216825 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664dd73-5481-436c-b2b5-84887c71940a service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:49.217620 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664dd73-5481-436c-b2b5-84887c71940a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:49.224428 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-4664dd73-5481-436c-b2b5-84887c71940a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:49.334256 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 951/1919] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:49.336175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23fa60-88b4-4465-a50f-8c0d7646cae7 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:49.336424 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23fa60-88b4-4465-a50f-8c0d7646cae7 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:49.336838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23fa60-88b4-4465-a50f-8c0d7646cae7 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:49.338829 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 952/1920] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jun 22 11:45:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:45:49.349814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613df145-3f96-47c0-a13a-bd49403db78a None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:49.349814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613df145-3f96-47c0-a13a-bd49403db78a None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:49.349814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613df145-3f96-47c0-a13a-bd49403db78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:49.369456 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-613df145-3f96-47c0-a13a-bd49403db78a None None] MFA Rules not processed for user `479a376dcac14c509c1c55f0046d88cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:49.371140 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-613df145-3f96-47c0-a13a-bd49403db78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:49.395886 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664dd73-5481-436c-b2b5-84887c71940a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:49.397454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664dd73-5481-436c-b2b5-84887c71940a service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:49.409610 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 969/1921] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:49 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:49.456600 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664dd73-5481-436c-b2b5-84887c71940a service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:49.469756 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664dd73-5481-436c-b2b5-84887c71940a service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:49.474592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2302e-35a9-43a8-8f26-60b3ef28771c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:49.475821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2302e-35a9-43a8-8f26-60b3ef28771c service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:49.476122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2302e-35a9-43a8-8f26-60b3ef28771c service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:49.476395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2302e-35a9-43a8-8f26-60b3ef28771c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:49.482493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 953/1922] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jun 22 11:45:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:49.496725 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2302e-35a9-43a8-8f26-60b3ef28771c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:49.498400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2302e-35a9-43a8-8f26-60b3ef28771c service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:49.511057 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 970/1923] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:49.548335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613df145-3f96-47c0-a13a-bd49403db78a None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:49.550163 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-613df145-3f96-47c0-a13a-bd49403db78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:49.560801 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2302e-35a9-43a8-8f26-60b3ef28771c service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:49.575626 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2302e-35a9-43a8-8f26-60b3ef28771c service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:49.581516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0297fc-f25d-45aa-9bc6-6d1aee04ed49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:49.582460 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0297fc-f25d-45aa-9bc6-6d1aee04ed49 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:49.583255 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0297fc-f25d-45aa-9bc6-6d1aee04ed49 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:49.584453 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0297fc-f25d-45aa-9bc6-6d1aee04ed49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:49.587161 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-1c0297fc-f25d-45aa-9bc6-6d1aee04ed49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:49.634124 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613df145-3f96-47c0-a13a-bd49403db78a None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:49.639129 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a7642-6803-4b66-a20d-b6a135634262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:49.640057 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a7642-6803-4b66-a20d-b6a135634262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:49.640338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a7642-6803-4b66-a20d-b6a135634262 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:49.640641 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a7642-6803-4b66-a20d-b6a135634262 admin admin] PATH_INFO: `/v3/users/035c6bb2d90e48c48a04a0ef095d511c` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:49.641941 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a7642-6803-4b66-a20d-b6a135634262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=035c6bb2d90e48c48a04a0ef095d511c)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:49.644584 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a7642-6803-4b66-a20d-b6a135634262 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:49.670234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:49.727736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ee5a7642-6803-4b66-a20d-b6a135634262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '035c6bb2d90e48c48a04a0ef095d511c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:49.729701 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:49.736580 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0297fc-f25d-45aa-9bc6-6d1aee04ed49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:49.738036 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0297fc-f25d-45aa-9bc6-6d1aee04ed49 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:49.746458 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ee5a7642-6803-4b66-a20d-b6a135634262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '035c6bb2d90e48c48a04a0ef095d511c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:49.747689 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 971/1924] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:49 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:49.748364 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0297fc-f25d-45aa-9bc6-6d1aee04ed49 service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:49.749728 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-1c0297fc-f25d-45aa-9bc6-6d1aee04ed49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:49.765866 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0297fc-f25d-45aa-9bc6-6d1aee04ed49 service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:49.773763 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ee5a7642-6803-4b66-a20d-b6a135634262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '035c6bb2d90e48c48a04a0ef095d511c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:49.774509 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ee5a7642-6803-4b66-a20d-b6a135634262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '035c6bb2d90e48c48a04a0ef095d511c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:49.787461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e716b57-5d80-40a6-8200-910bd91e78e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:49.788196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e716b57-5d80-40a6-8200-910bd91e78e0 service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:49.788637 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e716b57-5d80-40a6-8200-910bd91e78e0 service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:49.789204 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e716b57-5d80-40a6-8200-910bd91e78e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:49.792299 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ee5a7642-6803-4b66-a20d-b6a135634262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '035c6bb2d90e48c48a04a0ef095d511c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:49.801375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e716b57-5d80-40a6-8200-910bd91e78e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:49.803139 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e716b57-5d80-40a6-8200-910bd91e78e0 service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:49.803918 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 954/1925] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:49 2022] DELETE /identity/v3/users/035c6bb2d90e48c48a04a0ef095d511c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:49.804767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a7642-6803-4b66-a20d-b6a135634262 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:49.809261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ee5a7642-6803-4b66-a20d-b6a135634262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:49.816828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 972/1926] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:49.818773 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76bd0a-a5e8-4135-a91a-e63585a30b8f None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:49.819067 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76bd0a-a5e8-4135-a91a-e63585a30b8f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:49.819469 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76bd0a-a5e8-4135-a91a-e63585a30b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:49.852041 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-0d76bd0a-a5e8-4135-a91a-e63585a30b8f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:49.854355 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-0d76bd0a-a5e8-4135-a91a-e63585a30b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:49.931546 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a7642-6803-4b66-a20d-b6a135634262 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:49.955657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a6110-05dc-402e-a327-d10b3fd7b442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:49.961011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a6110-05dc-402e-a327-d10b3fd7b442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:49.961319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a6110-05dc-402e-a327-d10b3fd7b442 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:49.961591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a6110-05dc-402e-a327-d10b3fd7b442 admin admin] PATH_INFO: `/v3/users/f859eae147354580875085d05f98e482` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:49.965975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 973/1927] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:49.965975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76bd0a-a5e8-4135-a91a-e63585a30b8f None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:49.967903 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a6110-05dc-402e-a327-d10b3fd7b442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f859eae147354580875085d05f98e482)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:49.972827 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a6110-05dc-402e-a327-d10b3fd7b442 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:49.975244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-0d76bd0a-a5e8-4135-a91a-e63585a30b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:50.023931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76bd0a-a5e8-4135-a91a-e63585a30b8f None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:50.060845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ed0a4-45a3-48ad-a0b9-b5e4b4c1dff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:50.060845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ed0a4-45a3-48ad-a0b9-b5e4b4c1dff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:50.060845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ed0a4-45a3-48ad-a0b9-b5e4b4c1dff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:50.060845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ed0a4-45a3-48ad-a0b9-b5e4b4c1dff8 admin admin] PATH_INFO: `/v3/projects/8ca292141909417fa92590a10ddabeaf` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:50.061224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ed0a4-45a3-48ad-a0b9-b5e4b4c1dff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ca292141909417fa92590a10ddabeaf)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:50.069320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ed0a4-45a3-48ad-a0b9-b5e4b4c1dff8 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:50.190977 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-545a6110-05dc-402e-a327-d10b3fd7b442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f859eae147354580875085d05f98e482'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:50.199787 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d11ed0a4-45a3-48ad-a0b9-b5e4b4c1dff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ca292141909417fa92590a10ddabeaf', 'request_id': 'req-d11ed0a4-45a3-48ad-a0b9-b5e4b4c1dff8'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:50.205029 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-545a6110-05dc-402e-a327-d10b3fd7b442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f859eae147354580875085d05f98e482'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:50.208605 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d11ed0a4-45a3-48ad-a0b9-b5e4b4c1dff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ca292141909417fa92590a10ddabeaf', 'request_id': 'req-d11ed0a4-45a3-48ad-a0b9-b5e4b4c1dff8'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:50.214308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-545a6110-05dc-402e-a327-d10b3fd7b442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f859eae147354580875085d05f98e482'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:50.215137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-545a6110-05dc-402e-a327-d10b3fd7b442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f859eae147354580875085d05f98e482'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:50.218710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d11ed0a4-45a3-48ad-a0b9-b5e4b4c1dff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ca292141909417fa92590a10ddabeaf', 'request_id': 'req-d11ed0a4-45a3-48ad-a0b9-b5e4b4c1dff8'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:50.224492 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-545a6110-05dc-402e-a327-d10b3fd7b442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f859eae147354580875085d05f98e482'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:50.232354 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 955/1928] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:49 2022] DELETE /identity/v3/users/f859eae147354580875085d05f98e482 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:50.233245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a6110-05dc-402e-a327-d10b3fd7b442 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:50.235822 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-545a6110-05dc-402e-a327-d10b3fd7b442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:50.238854 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d11ed0a4-45a3-48ad-a0b9-b5e4b4c1dff8 admin admin] The token cache is being invalidate because project 8ca292141909417fa92590a10ddabeaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:50.239297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d11ed0a4-45a3-48ad-a0b9-b5e4b4c1dff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:50.240114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d11ed0a4-45a3-48ad-a0b9-b5e4b4c1dff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:50.242013 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 974/1929] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:49 2022] DELETE /identity/v3/projects/8ca292141909417fa92590a10ddabeaf => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:50.244444 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a4c47-6566-4568-bc3c-a3f45a0609d2 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:50.244812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a4c47-6566-4568-bc3c-a3f45a0609d2 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:50.245152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a4c47-6566-4568-bc3c-a3f45a0609d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:50.271223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-dd0a4c47-6566-4568-bc3c-a3f45a0609d2 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:50.272873 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-dd0a4c47-6566-4568-bc3c-a3f45a0609d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:50.301308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:50.307296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a6110-05dc-402e-a327-d10b3fd7b442 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:50.312655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515a8c6-d135-46f5-868b-fc5c17ec2094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:50.313809 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515a8c6-d135-46f5-868b-fc5c17ec2094 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:50.314795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515a8c6-d135-46f5-868b-fc5c17ec2094 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:50.315461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515a8c6-d135-46f5-868b-fc5c17ec2094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:50.317787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2515a8c6-d135-46f5-868b-fc5c17ec2094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:50.318977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:50.354652 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 975/1930] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:50.354652 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba6e22-fc9c-40c4-bb2d-9eaa390239be None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:50.354652 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba6e22-fc9c-40c4-bb2d-9eaa390239be None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:50.354652 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba6e22-fc9c-40c4-bb2d-9eaa390239be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:50.372939 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f4ba6e22-fc9c-40c4-bb2d-9eaa390239be None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:50.374146 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f4ba6e22-fc9c-40c4-bb2d-9eaa390239be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:50.382508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 976/1931] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:50.383465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba6e22-fc9c-40c4-bb2d-9eaa390239be None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:50.385636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f4ba6e22-fc9c-40c4-bb2d-9eaa390239be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:50.397689 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba6e22-fc9c-40c4-bb2d-9eaa390239be None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:50.401608 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3c7bd-8dbd-4956-a37e-27bd3cc30549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:50.402278 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3c7bd-8dbd-4956-a37e-27bd3cc30549 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:50.402591 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3c7bd-8dbd-4956-a37e-27bd3cc30549 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:50.402865 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3c7bd-8dbd-4956-a37e-27bd3cc30549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:50.405671 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515a8c6-d135-46f5-868b-fc5c17ec2094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:50.407661 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515a8c6-d135-46f5-868b-fc5c17ec2094 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:50.413255 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c3c7bd-8dbd-4956-a37e-27bd3cc30549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:50.414678 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c3c7bd-8dbd-4956-a37e-27bd3cc30549 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:50.419549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 956/1932] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:50.421126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6db5c-cada-4aca-8384-aa5f22a06aef None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:50.421478 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6db5c-cada-4aca-8384-aa5f22a06aef None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:50.421721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6db5c-cada-4aca-8384-aa5f22a06aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:50.426273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 977/1933] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:50.440476 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-e6d6db5c-cada-4aca-8384-aa5f22a06aef None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:50.441697 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e6d6db5c-cada-4aca-8384-aa5f22a06aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:50.450028 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 957/1934] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:50.454234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3c7bd-8dbd-4956-a37e-27bd3cc30549 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:50.464180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3c7bd-8dbd-4956-a37e-27bd3cc30549 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:50.469418 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1966ad-14b4-4561-b69c-00da31d5d2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:50.470145 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1966ad-14b4-4561-b69c-00da31d5d2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:50.470532 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1966ad-14b4-4561-b69c-00da31d5d2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:50.470808 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1966ad-14b4-4561-b69c-00da31d5d2d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:50.471559 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1966ad-14b4-4561-b69c-00da31d5d2d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:50.471865 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1966ad-14b4-4561-b69c-00da31d5d2d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:50.473522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1966ad-14b4-4561-b69c-00da31d5d2d4 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:50.494173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 978/1935] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:45:50.499470 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6db5c-cada-4aca-8384-aa5f22a06aef None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:50.511294 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6db5c-cada-4aca-8384-aa5f22a06aef None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:50.517313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c0c62-66d7-4827-a497-f0d888357080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:50.518530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c0c62-66d7-4827-a497-f0d888357080 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:50.519032 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c0c62-66d7-4827-a497-f0d888357080 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:50.519319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0c0c62-66d7-4827-a497-f0d888357080 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:50.520788 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c0c62-66d7-4827-a497-f0d888357080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:50.523034 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0c0c62-66d7-4827-a497-f0d888357080 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:50.565074 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 958/1936] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:50 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:50.566236 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1966ad-14b4-4561-b69c-00da31d5d2d4 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:50.578393 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1966ad-14b4-4561-b69c-00da31d5d2d4 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:50.640170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4ad8c-0d61-4369-aa99-888d3b92b215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:50.640170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4ad8c-0d61-4369-aa99-888d3b92b215 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:50.640474 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4ad8c-0d61-4369-aa99-888d3b92b215 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:50.641358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4ad8c-0d61-4369-aa99-888d3b92b215 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:50.642289 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a4ad8c-0d61-4369-aa99-888d3b92b215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:50.644300 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a4ad8c-0d61-4369-aa99-888d3b92b215 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:50.666608 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 979/1937] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:50 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:45:50.670864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c0c62-66d7-4827-a497-f0d888357080 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:50.682617 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0c0c62-66d7-4827-a497-f0d888357080 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:50.690040 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592d779-9d48-419b-ab92-6eef04e81b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:50.691383 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592d779-9d48-419b-ab92-6eef04e81b22 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:50.691501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592d779-9d48-419b-ab92-6eef04e81b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:50.691882 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592d779-9d48-419b-ab92-6eef04e81b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:50.692887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592d779-9d48-419b-ab92-6eef04e81b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:50.694775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592d779-9d48-419b-ab92-6eef04e81b22 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:50.715583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 959/1938] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:45:50.720386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4ad8c-0d61-4369-aa99-888d3b92b215 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:50.733787 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4ad8c-0d61-4369-aa99-888d3b92b215 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:50.742177 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cbbb2-a7b0-4dd6-a9b4-ddf516cf7081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:50.742876 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cbbb2-a7b0-4dd6-a9b4-ddf516cf7081 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:50.743166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cbbb2-a7b0-4dd6-a9b4-ddf516cf7081 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:50.743650 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cbbb2-a7b0-4dd6-a9b4-ddf516cf7081 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:50.744536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cbbb2-a7b0-4dd6-a9b4-ddf516cf7081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:50.746190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cbbb2-a7b0-4dd6-a9b4-ddf516cf7081 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:50.758199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 980/1939] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:50.764111 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592d779-9d48-419b-ab92-6eef04e81b22 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:50.775522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592d779-9d48-419b-ab92-6eef04e81b22 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:50.780339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3820cc6-82cf-43b2-bde1-74597452ac51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:50.781296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3820cc6-82cf-43b2-bde1-74597452ac51 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:50.782121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3820cc6-82cf-43b2-bde1-74597452ac51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:50.782398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3820cc6-82cf-43b2-bde1-74597452ac51 admin admin] PATH_INFO: `/v3/projects/d75d8e962e5c4c6583430bf34ba03210/users/9b7d95e108c84b2b84a95a7c10132317/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:50.802988 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3820cc6-82cf-43b2-bde1-74597452ac51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75d8e962e5c4c6583430bf34ba03210, user_id=9b7d95e108c84b2b84a95a7c10132317, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:50.805787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3820cc6-82cf-43b2-bde1-74597452ac51 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:50.823780 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 960/1940] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:50 2022] PUT /identity/v3/projects/d75d8e962e5c4c6583430bf34ba03210/users/9b7d95e108c84b2b84a95a7c10132317/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:50.835534 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f67e1e-06a9-4687-97ce-9be79410caa4 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:50.835836 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f67e1e-06a9-4687-97ce-9be79410caa4 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:50.836112 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f67e1e-06a9-4687-97ce-9be79410caa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:50.871741 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-b0f67e1e-06a9-4687-97ce-9be79410caa4 None None] MFA Rules not processed for user `9b7d95e108c84b2b84a95a7c10132317`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:50.873457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b0f67e1e-06a9-4687-97ce-9be79410caa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:51.001513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 981/1941] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:51.013422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3820cc6-82cf-43b2-bde1-74597452ac51 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:51.015174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f3820cc6-82cf-43b2-bde1-74597452ac51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:51.056882 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f67e1e-06a9-4687-97ce-9be79410caa4 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:51.058981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b0f67e1e-06a9-4687-97ce-9be79410caa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:51.081183 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:51.099936 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3820cc6-82cf-43b2-bde1-74597452ac51 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:51.103837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606f90f-6eb5-4588-a831-c96f7665487d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:51.104650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606f90f-6eb5-4588-a831-c96f7665487d service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:51.104998 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606f90f-6eb5-4588-a831-c96f7665487d service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:51.105371 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606f90f-6eb5-4588-a831-c96f7665487d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:51.115674 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3606f90f-6eb5-4588-a831-c96f7665487d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:51.117536 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3606f90f-6eb5-4588-a831-c96f7665487d service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:51.129098 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 961/1942] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:45:51.129804 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606f90f-6eb5-4588-a831-c96f7665487d service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:51.131170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-3606f90f-6eb5-4588-a831-c96f7665487d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:51.159947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f67e1e-06a9-4687-97ce-9be79410caa4 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:51.165686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9fb98-11d8-43fe-8fb8-fcc5f1ae64d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:51.166633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9fb98-11d8-43fe-8fb8-fcc5f1ae64d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:51.166909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9fb98-11d8-43fe-8fb8-fcc5f1ae64d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:51.167186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9fb98-11d8-43fe-8fb8-fcc5f1ae64d3 admin admin] PATH_INFO: `/v3/projects/a317fc7813824b0c94196bdf6e8ae81a` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:51.168527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9fb98-11d8-43fe-8fb8-fcc5f1ae64d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a317fc7813824b0c94196bdf6e8ae81a)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:51.170430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9fb98-11d8-43fe-8fb8-fcc5f1ae64d3 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:51.180545 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:51.214216 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606f90f-6eb5-4588-a831-c96f7665487d service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:51.222106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076d439-5498-448f-8473-f91a045e3ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:51.222106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076d439-5498-448f-8473-f91a045e3ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:51.222106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076d439-5498-448f-8473-f91a045e3ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:51.222479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076d439-5498-448f-8473-f91a045e3ea3 admin admin] PATH_INFO: `/v3/projects/a642d4e26a324e8fa0bdd46543b04b26` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:51.224204 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076d439-5498-448f-8473-f91a045e3ea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a642d4e26a324e8fa0bdd46543b04b26)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:51.226462 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076d439-5498-448f-8473-f91a045e3ea3 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:51.244275 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-37e9fb98-11d8-43fe-8fb8-fcc5f1ae64d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a317fc7813824b0c94196bdf6e8ae81a', 'request_id': 'req-37e9fb98-11d8-43fe-8fb8-fcc5f1ae64d3'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:51.253122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-37e9fb98-11d8-43fe-8fb8-fcc5f1ae64d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a317fc7813824b0c94196bdf6e8ae81a', 'request_id': 'req-37e9fb98-11d8-43fe-8fb8-fcc5f1ae64d3'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:51.267547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-37e9fb98-11d8-43fe-8fb8-fcc5f1ae64d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a317fc7813824b0c94196bdf6e8ae81a', 'request_id': 'req-37e9fb98-11d8-43fe-8fb8-fcc5f1ae64d3'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:51.295658 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-37e9fb98-11d8-43fe-8fb8-fcc5f1ae64d3 admin admin] The token cache is being invalidate because project a317fc7813824b0c94196bdf6e8ae81a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:51.296375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-37e9fb98-11d8-43fe-8fb8-fcc5f1ae64d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:51.299442 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-37e9fb98-11d8-43fe-8fb8-fcc5f1ae64d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:51.306342 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 982/1943] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:51 2022] DELETE /identity/v3/projects/a317fc7813824b0c94196bdf6e8ae81a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:51.311119 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9fb98-11d8-43fe-8fb8-fcc5f1ae64d3 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:51.314068 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-37e9fb98-11d8-43fe-8fb8-fcc5f1ae64d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:51.339339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a076d439-5498-448f-8473-f91a045e3ea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a642d4e26a324e8fa0bdd46543b04b26', 'request_id': 'req-a076d439-5498-448f-8473-f91a045e3ea3'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:51.358865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a076d439-5498-448f-8473-f91a045e3ea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a642d4e26a324e8fa0bdd46543b04b26', 'request_id': 'req-a076d439-5498-448f-8473-f91a045e3ea3'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:51.379212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a076d439-5498-448f-8473-f91a045e3ea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a642d4e26a324e8fa0bdd46543b04b26', 'request_id': 'req-a076d439-5498-448f-8473-f91a045e3ea3'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:51.413706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a076d439-5498-448f-8473-f91a045e3ea3 admin admin] The token cache is being invalidate because project a642d4e26a324e8fa0bdd46543b04b26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:51.414312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a076d439-5498-448f-8473-f91a045e3ea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:51.430062 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a076d439-5498-448f-8473-f91a045e3ea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:51.441754 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 962/1944] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:51 2022] DELETE /identity/v3/projects/a642d4e26a324e8fa0bdd46543b04b26 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:51.447396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868f058-fce2-4d05-8588-a47302bd4718 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:51.449007 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868f058-fce2-4d05-8588-a47302bd4718 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:51.451848 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868f058-fce2-4d05-8588-a47302bd4718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:51.500944 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-6868f058-fce2-4d05-8588-a47302bd4718 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:51.503297 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-6868f058-fce2-4d05-8588-a47302bd4718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:51.541787 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9fb98-11d8-43fe-8fb8-fcc5f1ae64d3 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:51.610884 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 963/1945] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:51.612565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26068b9-87dc-4b6f-8bfc-09bdc0e7dcde None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:51.612863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26068b9-87dc-4b6f-8bfc-09bdc0e7dcde None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:51.613150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26068b9-87dc-4b6f-8bfc-09bdc0e7dcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:51.655679 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-e26068b9-87dc-4b6f-8bfc-09bdc0e7dcde None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:51.657673 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e26068b9-87dc-4b6f-8bfc-09bdc0e7dcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:51.658728 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:51.671307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 964/1946] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:51.675929 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26068b9-87dc-4b6f-8bfc-09bdc0e7dcde None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:51.689059 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f0fdf-9f04-4f8d-b345-260404bda592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:51.690265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f0fdf-9f04-4f8d-b345-260404bda592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:51.690579 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f0fdf-9f04-4f8d-b345-260404bda592 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:51.691125 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f0fdf-9f04-4f8d-b345-260404bda592 admin admin] PATH_INFO: `/v3/users/d0ac1142757242d5b0f240b671769639` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:51.691494 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26068b9-87dc-4b6f-8bfc-09bdc0e7dcde None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:51.692781 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f0fdf-9f04-4f8d-b345-260404bda592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0ac1142757242d5b0f240b671769639)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:51.694802 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f0fdf-9f04-4f8d-b345-260404bda592 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:51.709537 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad4608-acfd-4f64-9e2a-e83f2827b7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:51.710382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad4608-acfd-4f64-9e2a-e83f2827b7ac admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:51.710698 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad4608-acfd-4f64-9e2a-e83f2827b7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:51.711220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad4608-acfd-4f64-9e2a-e83f2827b7ac admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:51.712301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad4608-acfd-4f64-9e2a-e83f2827b7ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:51.712622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad4608-acfd-4f64-9e2a-e83f2827b7ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:51.715872 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad4608-acfd-4f64-9e2a-e83f2827b7ac admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:51.729565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 965/1947] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:45:51.734262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad4608-acfd-4f64-9e2a-e83f2827b7ac admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:51.747624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad4608-acfd-4f64-9e2a-e83f2827b7ac admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:51.748723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-317f0fdf-9f04-4f8d-b345-260404bda592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0ac1142757242d5b0f240b671769639'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:51.752445 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5decd96-3394-4ab0-8ea1-f984c2e46a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:51.753100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5decd96-3394-4ab0-8ea1-f984c2e46a0d admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:51.753388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5decd96-3394-4ab0-8ea1-f984c2e46a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:51.753692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5decd96-3394-4ab0-8ea1-f984c2e46a0d admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:51.754391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5decd96-3394-4ab0-8ea1-f984c2e46a0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:51.756357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5decd96-3394-4ab0-8ea1-f984c2e46a0d admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:51.760947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-317f0fdf-9f04-4f8d-b345-260404bda592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0ac1142757242d5b0f240b671769639'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:51.770007 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-317f0fdf-9f04-4f8d-b345-260404bda592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0ac1142757242d5b0f240b671769639'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:51.777685 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-317f0fdf-9f04-4f8d-b345-260404bda592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0ac1142757242d5b0f240b671769639'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:51.778354 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-317f0fdf-9f04-4f8d-b345-260404bda592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0ac1142757242d5b0f240b671769639'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:51.785017 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 966/1948] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:51 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:45:51.785193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 983/1949] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:45:51 2022] DELETE /identity/v3/users/d0ac1142757242d5b0f240b671769639 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:51.792403 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5decd96-3394-4ab0-8ea1-f984c2e46a0d admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:51.794170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e5decd96-3394-4ab0-8ea1-f984c2e46a0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:51.822014 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:51.862022 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5decd96-3394-4ab0-8ea1-f984c2e46a0d admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:51.927061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba284b35-d3a3-4cad-b275-4a14760adb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:51.927950 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba284b35-d3a3-4cad-b275-4a14760adb8a admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:51.928452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba284b35-d3a3-4cad-b275-4a14760adb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:51.929080 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba284b35-d3a3-4cad-b275-4a14760adb8a admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:51.930515 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba284b35-d3a3-4cad-b275-4a14760adb8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:51.932724 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba284b35-d3a3-4cad-b275-4a14760adb8a admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:51.962824 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 967/1950] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:51 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:51.965004 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f0fdf-9f04-4f8d-b345-260404bda592 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:51.976632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f0fdf-9f04-4f8d-b345-260404bda592 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:51.983747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99291d1f-d3cc-46ea-8102-cfb4ce7697ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:51.984554 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99291d1f-d3cc-46ea-8102-cfb4ce7697ef admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:51.984934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99291d1f-d3cc-46ea-8102-cfb4ce7697ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:51.985305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99291d1f-d3cc-46ea-8102-cfb4ce7697ef admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:51.986137 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99291d1f-d3cc-46ea-8102-cfb4ce7697ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:51.988417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99291d1f-d3cc-46ea-8102-cfb4ce7697ef admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:52.000721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 984/1951] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:52.004476 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba284b35-d3a3-4cad-b275-4a14760adb8a admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:52.019665 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba284b35-d3a3-4cad-b275-4a14760adb8a admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:52.025999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadeebb-fc99-4052-8396-2a150c58ce41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:52.026758 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadeebb-fc99-4052-8396-2a150c58ce41 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:52.027108 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadeebb-fc99-4052-8396-2a150c58ce41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:52.027496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadeebb-fc99-4052-8396-2a150c58ce41 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:52.028307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadeebb-fc99-4052-8396-2a150c58ce41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:52.030134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadeebb-fc99-4052-8396-2a150c58ce41 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:52.047288 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 968/1952] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:52.048558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99291d1f-d3cc-46ea-8102-cfb4ce7697ef admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:52.060554 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99291d1f-d3cc-46ea-8102-cfb4ce7697ef admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:52.080130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11c848-26cf-4687-b09b-7bbf7774ee17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:52.081114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11c848-26cf-4687-b09b-7bbf7774ee17 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:52.081447 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11c848-26cf-4687-b09b-7bbf7774ee17 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:52.081800 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11c848-26cf-4687-b09b-7bbf7774ee17 admin admin] PATH_INFO: `/v3/projects/154ca230a6744530a3c3eab7653673f1/users/f69ea1b2d30a4359aa41d87ab8859b00/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:52.095874 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadeebb-fc99-4052-8396-2a150c58ce41 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:52.098261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-aaadeebb-fc99-4052-8396-2a150c58ce41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:52.114809 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11c848-26cf-4687-b09b-7bbf7774ee17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=154ca230a6744530a3c3eab7653673f1, user_id=f69ea1b2d30a4359aa41d87ab8859b00, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:52.115807 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadeebb-fc99-4052-8396-2a150c58ce41 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:52.120251 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11c848-26cf-4687-b09b-7bbf7774ee17 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:52.120786 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b55c10-5781-49d2-9f12-f3e6b920145f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:52.121961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b55c10-5781-49d2-9f12-f3e6b920145f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:52.122259 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b55c10-5781-49d2-9f12-f3e6b920145f admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:52.122534 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b55c10-5781-49d2-9f12-f3e6b920145f admin admin] PATH_INFO: `/v3/projects/e785f4565e504d0caef18c405c8cfa6c` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:52.123897 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b55c10-5781-49d2-9f12-f3e6b920145f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e785f4565e504d0caef18c405c8cfa6c)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:52.126537 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b55c10-5781-49d2-9f12-f3e6b920145f admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:52.147615 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 985/1953] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:52 2022] PUT /identity/v3/projects/154ca230a6744530a3c3eab7653673f1/users/f69ea1b2d30a4359aa41d87ab8859b00/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:52.154589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26025cda-fa9a-41a2-aa2f-705615b43633 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:52.155008 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26025cda-fa9a-41a2-aa2f-705615b43633 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:52.155506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26025cda-fa9a-41a2-aa2f-705615b43633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:52.189158 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-90b55c10-5781-49d2-9f12-f3e6b920145f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e785f4565e504d0caef18c405c8cfa6c', 'request_id': 'req-90b55c10-5781-49d2-9f12-f3e6b920145f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:52.191903 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-26025cda-fa9a-41a2-aa2f-705615b43633 None None] MFA Rules not processed for user `f69ea1b2d30a4359aa41d87ab8859b00`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:52.193244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-26025cda-fa9a-41a2-aa2f-705615b43633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:52.198525 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-90b55c10-5781-49d2-9f12-f3e6b920145f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e785f4565e504d0caef18c405c8cfa6c', 'request_id': 'req-90b55c10-5781-49d2-9f12-f3e6b920145f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:52.209200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-90b55c10-5781-49d2-9f12-f3e6b920145f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e785f4565e504d0caef18c405c8cfa6c', 'request_id': 'req-90b55c10-5781-49d2-9f12-f3e6b920145f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:52.221132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-90b55c10-5781-49d2-9f12-f3e6b920145f admin admin] The token cache is being invalidate because project e785f4565e504d0caef18c405c8cfa6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:45:52.221413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-90b55c10-5781-49d2-9f12-f3e6b920145f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:52.222081 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-90b55c10-5781-49d2-9f12-f3e6b920145f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:45:52.223819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 969/1954] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:45:52 2022] DELETE /identity/v3/projects/e785f4565e504d0caef18c405c8cfa6c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:52.228591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60a225-a628-42ee-a9f0-9877e620d349 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:52.229352 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60a225-a628-42ee-a9f0-9877e620d349 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:52.229352 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60a225-a628-42ee-a9f0-9877e620d349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:52.250896 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-6b60a225-a628-42ee-a9f0-9877e620d349 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:52.252382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-6b60a225-a628-42ee-a9f0-9877e620d349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:52.284856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:52.290816 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 986/1955] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:45:52.302676 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26025cda-fa9a-41a2-aa2f-705615b43633 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:52.304070 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-26025cda-fa9a-41a2-aa2f-705615b43633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:52.327747 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 970/1956] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:52.329350 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47b083-d212-4cde-b543-9c9b1a873c41 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:52.329634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47b083-d212-4cde-b543-9c9b1a873c41 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:52.329910 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47b083-d212-4cde-b543-9c9b1a873c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:52.336226 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:52.349073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-4f47b083-d212-4cde-b543-9c9b1a873c41 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:52.350304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-4f47b083-d212-4cde-b543-9c9b1a873c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:52.358294 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 971/1957] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:52.362954 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47b083-d212-4cde-b543-9c9b1a873c41 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:52.368395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26025cda-fa9a-41a2-aa2f-705615b43633 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:52.372080 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77454fe-0129-4d1f-8258-db3f1d246c7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:52.372215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47b083-d212-4cde-b543-9c9b1a873c41 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:52.372732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77454fe-0129-4d1f-8258-db3f1d246c7f service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:52.373014 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77454fe-0129-4d1f-8258-db3f1d246c7f service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:52.373294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77454fe-0129-4d1f-8258-db3f1d246c7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:52.377292 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb48efb-a125-4140-a33e-3b1a8f6d8811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:52.377966 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb48efb-a125-4140-a33e-3b1a8f6d8811 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:52.378245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb48efb-a125-4140-a33e-3b1a8f6d8811 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:52.378519 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb48efb-a125-4140-a33e-3b1a8f6d8811 admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:52.379235 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb48efb-a125-4140-a33e-3b1a8f6d8811 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:45:52.379569 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb48efb-a125-4140-a33e-3b1a8f6d8811 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:52.381176 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb48efb-a125-4140-a33e-3b1a8f6d8811 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:52.382345 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77454fe-0129-4d1f-8258-db3f1d246c7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:52.383808 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77454fe-0129-4d1f-8258-db3f1d246c7f service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:52.393042 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 972/1958] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:45:52.396476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 987/1959] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:45:52.397256 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77454fe-0129-4d1f-8258-db3f1d246c7f service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:52.408875 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77454fe-0129-4d1f-8258-db3f1d246c7f service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:52.413540 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407124a1-5a40-4527-9994-1dd007e98301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:52.414191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407124a1-5a40-4527-9994-1dd007e98301 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:52.414475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407124a1-5a40-4527-9994-1dd007e98301 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:52.414751 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407124a1-5a40-4527-9994-1dd007e98301 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:52.415495 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407124a1-5a40-4527-9994-1dd007e98301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:52.417425 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407124a1-5a40-4527-9994-1dd007e98301 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:52.462707 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 988/1960] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:52 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:45:52.468809 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb48efb-a125-4140-a33e-3b1a8f6d8811 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:52.481236 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb48efb-a125-4140-a33e-3b1a8f6d8811 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:52.627803 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b29d05-648e-4cb2-8922-f9af020ea495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:52.628576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b29d05-648e-4cb2-8922-f9af020ea495 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:52.628861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b29d05-648e-4cb2-8922-f9af020ea495 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:52.629132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b29d05-648e-4cb2-8922-f9af020ea495 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:52.629855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b29d05-648e-4cb2-8922-f9af020ea495 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:52.631879 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b29d05-648e-4cb2-8922-f9af020ea495 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:52.658888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 973/1961] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:52 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:52.663006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407124a1-5a40-4527-9994-1dd007e98301 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:52.675065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407124a1-5a40-4527-9994-1dd007e98301 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:52.680746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288f6d5-752e-4361-aa71-d971e0a02efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:52.682184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288f6d5-752e-4361-aa71-d971e0a02efe admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:52.682742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288f6d5-752e-4361-aa71-d971e0a02efe admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:52.683095 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288f6d5-752e-4361-aa71-d971e0a02efe admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:52.684767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288f6d5-752e-4361-aa71-d971e0a02efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:52.688260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288f6d5-752e-4361-aa71-d971e0a02efe admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:52.700775 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 989/1962] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:52.705599 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b29d05-648e-4cb2-8922-f9af020ea495 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:52.717880 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b29d05-648e-4cb2-8922-f9af020ea495 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:52.726722 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873d6fb-3530-46d0-8647-d40e353bb015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:52.727863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873d6fb-3530-46d0-8647-d40e353bb015 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:52.728188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873d6fb-3530-46d0-8647-d40e353bb015 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:52.728427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873d6fb-3530-46d0-8647-d40e353bb015 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:52.729555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873d6fb-3530-46d0-8647-d40e353bb015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:52.731739 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873d6fb-3530-46d0-8647-d40e353bb015 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:52.756096 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 974/1963] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:45:52.760388 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288f6d5-752e-4361-aa71-d971e0a02efe admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:52.794863 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288f6d5-752e-4361-aa71-d971e0a02efe admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:52.808488 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628020d-2072-4206-9246-2fd7cc4a06e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:52.810074 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628020d-2072-4206-9246-2fd7cc4a06e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:52.810074 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628020d-2072-4206-9246-2fd7cc4a06e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:52.810262 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628020d-2072-4206-9246-2fd7cc4a06e4 admin admin] PATH_INFO: `/v3/projects/91c645733c504bec8ca9b0f82ed5c237/users/eb71a8dc3bef478bba83e58384923ab8/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:52.829381 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628020d-2072-4206-9246-2fd7cc4a06e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91c645733c504bec8ca9b0f82ed5c237, user_id=eb71a8dc3bef478bba83e58384923ab8, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:52.832094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628020d-2072-4206-9246-2fd7cc4a06e4 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:52.866058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 990/1964] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:52 2022] PUT /identity/v3/projects/91c645733c504bec8ca9b0f82ed5c237/users/eb71a8dc3bef478bba83e58384923ab8/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:52.879182 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a04c5a-4046-407a-b817-41a2e54a1d72 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:52.879600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a04c5a-4046-407a-b817-41a2e54a1d72 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:52.879834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a04c5a-4046-407a-b817-41a2e54a1d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:52.910468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-44a04c5a-4046-407a-b817-41a2e54a1d72 None None] MFA Rules not processed for user `eb71a8dc3bef478bba83e58384923ab8`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:52.911920 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-44a04c5a-4046-407a-b817-41a2e54a1d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:52.982014 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 975/1965] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:45:52.987257 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628020d-2072-4206-9246-2fd7cc4a06e4 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:53.055622 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628020d-2072-4206-9246-2fd7cc4a06e4 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:53.060380 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c210803-a749-423d-b16d-a0a9cfcf3907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:53.061042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c210803-a749-423d-b16d-a0a9cfcf3907 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:53.061331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c210803-a749-423d-b16d-a0a9cfcf3907 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:53.061672 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c210803-a749-423d-b16d-a0a9cfcf3907 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:53.062311 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c210803-a749-423d-b16d-a0a9cfcf3907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:53.064844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c210803-a749-423d-b16d-a0a9cfcf3907 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:53.092022 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 991/1966] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:45:52 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:45:53.096377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a04c5a-4046-407a-b817-41a2e54a1d72 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:53.107095 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a04c5a-4046-407a-b817-41a2e54a1d72 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:53.126986 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:53.166494 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269423cd-5f6e-4247-af09-559585a6a9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:53.167163 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269423cd-5f6e-4247-af09-559585a6a9ec admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:53.167567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269423cd-5f6e-4247-af09-559585a6a9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:53.167972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269423cd-5f6e-4247-af09-559585a6a9ec admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:53.168715 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269423cd-5f6e-4247-af09-559585a6a9ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:53.170697 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269423cd-5f6e-4247-af09-559585a6a9ec admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:53.192359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 976/1967] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:45:53 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:45:53.196602 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c210803-a749-423d-b16d-a0a9cfcf3907 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:53.207657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c210803-a749-423d-b16d-a0a9cfcf3907 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:53.212424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b305f98-692e-4d1b-8daa-77c0798cbcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:53.213088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b305f98-692e-4d1b-8daa-77c0798cbcf9 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:53.213370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b305f98-692e-4d1b-8daa-77c0798cbcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:53.213649 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b305f98-692e-4d1b-8daa-77c0798cbcf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:53.214376 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b305f98-692e-4d1b-8daa-77c0798cbcf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:53.216010 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b305f98-692e-4d1b-8daa-77c0798cbcf9 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:53.226547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 992/1968] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:53.230598 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269423cd-5f6e-4247-af09-559585a6a9ec admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:53.240989 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269423cd-5f6e-4247-af09-559585a6a9ec admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:53.245535 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5dd943-0497-45ed-bba2-6b8608e32607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:53.246403 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5dd943-0497-45ed-bba2-6b8608e32607 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:53.246689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5dd943-0497-45ed-bba2-6b8608e32607 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:53.246961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5dd943-0497-45ed-bba2-6b8608e32607 admin admin] PATH_INFO: `/v3/projects/9a6d95600cab4b329b6eff2a9e61c6ac/users/c1c489335ef541408c65e72be71806cb/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:53.261991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5dd943-0497-45ed-bba2-6b8608e32607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a6d95600cab4b329b6eff2a9e61c6ac, user_id=c1c489335ef541408c65e72be71806cb, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:53.264690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5dd943-0497-45ed-bba2-6b8608e32607 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:53.281977 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 977/1969] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:53 2022] PUT /identity/v3/projects/9a6d95600cab4b329b6eff2a9e61c6ac/users/c1c489335ef541408c65e72be71806cb/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:53.286843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b305f98-692e-4d1b-8daa-77c0798cbcf9 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:53.304610 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b305f98-692e-4d1b-8daa-77c0798cbcf9 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:53.358931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:45:53.376061 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a3c48-66f5-4840-bf88-29844a336b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:53.377574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a3c48-66f5-4840-bf88-29844a336b8c admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:53.377574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a3c48-66f5-4840-bf88-29844a336b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:53.377574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a3c48-66f5-4840-bf88-29844a336b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:53.378394 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a3c48-66f5-4840-bf88-29844a336b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:53.380272 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a3c48-66f5-4840-bf88-29844a336b8c admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:53.390952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 993/1970] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:45:53.395796 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5dd943-0497-45ed-bba2-6b8608e32607 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:53.406966 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5dd943-0497-45ed-bba2-6b8608e32607 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:53.411667 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd3dfc-2871-4994-9dae-4aefef43080f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:53.412602 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd3dfc-2871-4994-9dae-4aefef43080f admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:53.412836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd3dfc-2871-4994-9dae-4aefef43080f admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:53.413112 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd3dfc-2871-4994-9dae-4aefef43080f admin admin] PATH_INFO: `/v3/projects/9a6d95600cab4b329b6eff2a9e61c6ac/users/c1c489335ef541408c65e72be71806cb/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:53.415109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd3dfc-2871-4994-9dae-4aefef43080f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a6d95600cab4b329b6eff2a9e61c6ac, user_id=c1c489335ef541408c65e72be71806cb, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:53.417790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd3dfc-2871-4994-9dae-4aefef43080f admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:53.429459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 978/1971] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:45:53 2022] PUT /identity/v3/projects/9a6d95600cab4b329b6eff2a9e61c6ac/users/c1c489335ef541408c65e72be71806cb/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:45:53.439280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f20a09-07fc-4dca-821a-a4d325872395 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:53.439692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f20a09-07fc-4dca-821a-a4d325872395 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:53.439978 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f20a09-07fc-4dca-821a-a4d325872395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:53.469676 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-81f20a09-07fc-4dca-821a-a4d325872395 None None] MFA Rules not processed for user `c1c489335ef541408c65e72be71806cb`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:45:53.471128 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-81f20a09-07fc-4dca-821a-a4d325872395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:45:53.551569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 994/1972] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:45:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:45:53.562342 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd3dfc-2871-4994-9dae-4aefef43080f admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:53.624121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd3dfc-2871-4994-9dae-4aefef43080f admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:53.627764 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a59b0e-1f7c-462d-94a5-443be0a8dde6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:53.628420 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a59b0e-1f7c-462d-94a5-443be0a8dde6 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:53.628705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a59b0e-1f7c-462d-94a5-443be0a8dde6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:53.628981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a59b0e-1f7c-462d-94a5-443be0a8dde6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:53.638591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a59b0e-1f7c-462d-94a5-443be0a8dde6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:53.640076 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a59b0e-1f7c-462d-94a5-443be0a8dde6 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:53.650834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 979/1973] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:45:53.758276 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f20a09-07fc-4dca-821a-a4d325872395 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:45:53.769754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f20a09-07fc-4dca-821a-a4d325872395 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:45:53.773558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69686b9-9bbf-4f57-a0b5-eabf94ffcea1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:45:53.774201 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69686b9-9bbf-4f57-a0b5-eabf94ffcea1 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:45:53.774511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69686b9-9bbf-4f57-a0b5-eabf94ffcea1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:45:53.774793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69686b9-9bbf-4f57-a0b5-eabf94ffcea1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:45:53.837000 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69686b9-9bbf-4f57-a0b5-eabf94ffcea1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:45:53.838477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69686b9-9bbf-4f57-a0b5-eabf94ffcea1 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:45:53.848726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 995/1974] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:45:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:01.629365 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a59b0e-1f7c-462d-94a5-443be0a8dde6 service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:01.631106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d3a59b0e-1f7c-462d-94a5-443be0a8dde6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:01.706919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a59b0e-1f7c-462d-94a5-443be0a8dde6 service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:01.711888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1906587-c55f-46f9-9d32-e81a6e995d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:01.712767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1906587-c55f-46f9-9d32-e81a6e995d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:01.713047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1906587-c55f-46f9-9d32-e81a6e995d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:01.713317 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1906587-c55f-46f9-9d32-e81a6e995d92 admin admin] PATH_INFO: `/v3/users/4b4d92c9354e429b876a006ce4d0307f` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:01.714519 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1906587-c55f-46f9-9d32-e81a6e995d92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b4d92c9354e429b876a006ce4d0307f)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:01.716498 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1906587-c55f-46f9-9d32-e81a6e995d92 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:01.739177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:01.768757 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-c1906587-c55f-46f9-9d32-e81a6e995d92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b4d92c9354e429b876a006ce4d0307f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:01.787152 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-c1906587-c55f-46f9-9d32-e81a6e995d92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b4d92c9354e429b876a006ce4d0307f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:01.788005 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69686b9-9bbf-4f57-a0b5-eabf94ffcea1 service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:01.792606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-a69686b9-9bbf-4f57-a0b5-eabf94ffcea1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:01.797931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-c1906587-c55f-46f9-9d32-e81a6e995d92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b4d92c9354e429b876a006ce4d0307f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:01.798637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-c1906587-c55f-46f9-9d32-e81a6e995d92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b4d92c9354e429b876a006ce4d0307f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:01.808275 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-c1906587-c55f-46f9-9d32-e81a6e995d92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b4d92c9354e429b876a006ce4d0307f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:01.809900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69686b9-9bbf-4f57-a0b5-eabf94ffcea1 service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:01.815219 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ebc06-ae80-43bd-baba-842909fa415b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:01.816119 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 980/1975] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:01 2022] DELETE /identity/v3/users/4b4d92c9354e429b876a006ce4d0307f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:01.816259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ebc06-ae80-43bd-baba-842909fa415b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:01.816498 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ebc06-ae80-43bd-baba-842909fa415b admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:01.816858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ebc06-ae80-43bd-baba-842909fa415b admin admin] PATH_INFO: `/v3/users/9b7d95e108c84b2b84a95a7c10132317` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:01.820035 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ebc06-ae80-43bd-baba-842909fa415b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b7d95e108c84b2b84a95a7c10132317)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:01.820823 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957ec3d-5a8d-453e-8f2b-3204bf33afb2 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:01.821190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957ec3d-5a8d-453e-8f2b-3204bf33afb2 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:01.821543 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957ec3d-5a8d-453e-8f2b-3204bf33afb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:01.822292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ebc06-ae80-43bd-baba-842909fa415b admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:01.845838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-8957ec3d-5a8d-453e-8f2b-3204bf33afb2 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:01.847183 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-8957ec3d-5a8d-453e-8f2b-3204bf33afb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:01.883133 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-091ebc06-ae80-43bd-baba-842909fa415b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b7d95e108c84b2b84a95a7c10132317'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:01.891582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:01.898080 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-091ebc06-ae80-43bd-baba-842909fa415b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b7d95e108c84b2b84a95a7c10132317'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:01.906595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-091ebc06-ae80-43bd-baba-842909fa415b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b7d95e108c84b2b84a95a7c10132317'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:01.914055 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-091ebc06-ae80-43bd-baba-842909fa415b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b7d95e108c84b2b84a95a7c10132317'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:01.914713 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-091ebc06-ae80-43bd-baba-842909fa415b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b7d95e108c84b2b84a95a7c10132317'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:01.914820 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 981/1976] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:01.921081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 996/1977] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:01 2022] DELETE /identity/v3/users/9b7d95e108c84b2b84a95a7c10132317 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:01.925036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957ec3d-5a8d-453e-8f2b-3204bf33afb2 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:01.925801 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016ec82-d1b4-449f-aca1-4e3b4c2f82a9 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:01.926120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016ec82-d1b4-449f-aca1-4e3b4c2f82a9 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:01.926405 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016ec82-d1b4-449f-aca1-4e3b4c2f82a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:01.927062 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-8957ec3d-5a8d-453e-8f2b-3204bf33afb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:01.944353 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-e016ec82-d1b4-449f-aca1-4e3b4c2f82a9 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:01.945587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e016ec82-d1b4-449f-aca1-4e3b4c2f82a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:01.993553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957ec3d-5a8d-453e-8f2b-3204bf33afb2 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:02.009440 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 997/1978] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:02.019920 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016ec82-d1b4-449f-aca1-4e3b4c2f82a9 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:02.021280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e016ec82-d1b4-449f-aca1-4e3b4c2f82a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:02.046622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047c030-ae3a-4b01-90d6-26a68757546f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:02.047272 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047c030-ae3a-4b01-90d6-26a68757546f service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:02.047583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047c030-ae3a-4b01-90d6-26a68757546f service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:02.047883 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047c030-ae3a-4b01-90d6-26a68757546f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:02.049690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a047c030-ae3a-4b01-90d6-26a68757546f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:02.059253 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:02.061826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047c030-ae3a-4b01-90d6-26a68757546f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:02.063797 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047c030-ae3a-4b01-90d6-26a68757546f service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:02.075792 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 982/1979] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:46:02.079925 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016ec82-d1b4-449f-aca1-4e3b4c2f82a9 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:02.085502 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335f71d-9437-4500-8a8d-69b20bf5342a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:02.086349 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335f71d-9437-4500-8a8d-69b20bf5342a service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:02.087008 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335f71d-9437-4500-8a8d-69b20bf5342a service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:02.087459 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335f71d-9437-4500-8a8d-69b20bf5342a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:02.098183 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335f71d-9437-4500-8a8d-69b20bf5342a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:02.100055 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335f71d-9437-4500-8a8d-69b20bf5342a service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:02.112909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 998/1980] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:46:02.595997 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047c030-ae3a-4b01-90d6-26a68757546f service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:02.597411 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a047c030-ae3a-4b01-90d6-26a68757546f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:02.610491 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047c030-ae3a-4b01-90d6-26a68757546f service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:02.614908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb0eb4-a43d-4d63-bb91-05c1ec27d77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:02.616010 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb0eb4-a43d-4d63-bb91-05c1ec27d77c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:02.616390 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb0eb4-a43d-4d63-bb91-05c1ec27d77c admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:02.616767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb0eb4-a43d-4d63-bb91-05c1ec27d77c admin admin] PATH_INFO: `/v3/projects/eeddcf9cf4094ed78967714498b6eb4a` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:02.623152 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb0eb4-a43d-4d63-bb91-05c1ec27d77c admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeddcf9cf4094ed78967714498b6eb4a)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:02.625762 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb0eb4-a43d-4d63-bb91-05c1ec27d77c admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:02.670188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335f71d-9437-4500-8a8d-69b20bf5342a service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:02.672173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-c335f71d-9437-4500-8a8d-69b20bf5342a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:02.688930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4cbb0eb4-a43d-4d63-bb91-05c1ec27d77c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeddcf9cf4094ed78967714498b6eb4a', 'request_id': 'req-4cbb0eb4-a43d-4d63-bb91-05c1ec27d77c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:02.697185 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4cbb0eb4-a43d-4d63-bb91-05c1ec27d77c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeddcf9cf4094ed78967714498b6eb4a', 'request_id': 'req-4cbb0eb4-a43d-4d63-bb91-05c1ec27d77c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:02.698729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335f71d-9437-4500-8a8d-69b20bf5342a service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:02.707891 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4cbb0eb4-a43d-4d63-bb91-05c1ec27d77c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeddcf9cf4094ed78967714498b6eb4a', 'request_id': 'req-4cbb0eb4-a43d-4d63-bb91-05c1ec27d77c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:02.720775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4cbb0eb4-a43d-4d63-bb91-05c1ec27d77c admin admin] The token cache is being invalidate because project eeddcf9cf4094ed78967714498b6eb4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:02.721141 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4cbb0eb4-a43d-4d63-bb91-05c1ec27d77c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:02.721871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4cbb0eb4-a43d-4d63-bb91-05c1ec27d77c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:02.723528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 983/1981] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:02 2022] DELETE /identity/v3/projects/eeddcf9cf4094ed78967714498b6eb4a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:02.727332 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb0eb4-a43d-4d63-bb91-05c1ec27d77c admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:02.728782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-4cbb0eb4-a43d-4d63-bb91-05c1ec27d77c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:02.760398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a545ab1-1d74-419c-bd47-3c974cde1258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:02.761459 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a545ab1-1d74-419c-bd47-3c974cde1258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:02.761747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a545ab1-1d74-419c-bd47-3c974cde1258 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:02.762184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a545ab1-1d74-419c-bd47-3c974cde1258 admin admin] PATH_INFO: `/v3/projects/d75d8e962e5c4c6583430bf34ba03210` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:02.763230 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a545ab1-1d74-419c-bd47-3c974cde1258 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d75d8e962e5c4c6583430bf34ba03210)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:02.765236 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a545ab1-1d74-419c-bd47-3c974cde1258 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:02.792592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:02.809115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb0eb4-a43d-4d63-bb91-05c1ec27d77c admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:02.825464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-2a545ab1-1d74-419c-bd47-3c974cde1258 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd75d8e962e5c4c6583430bf34ba03210', 'request_id': 'req-2a545ab1-1d74-419c-bd47-3c974cde1258'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:02.831942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-2a545ab1-1d74-419c-bd47-3c974cde1258 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd75d8e962e5c4c6583430bf34ba03210', 'request_id': 'req-2a545ab1-1d74-419c-bd47-3c974cde1258'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:02.840410 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-2a545ab1-1d74-419c-bd47-3c974cde1258 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd75d8e962e5c4c6583430bf34ba03210', 'request_id': 'req-2a545ab1-1d74-419c-bd47-3c974cde1258'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:02.851182 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-2a545ab1-1d74-419c-bd47-3c974cde1258 admin admin] The token cache is being invalidate because project d75d8e962e5c4c6583430bf34ba03210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:02.851499 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-2a545ab1-1d74-419c-bd47-3c974cde1258 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:02.852132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-2a545ab1-1d74-419c-bd47-3c974cde1258 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:02.853679 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 999/1982] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:02 2022] DELETE /identity/v3/projects/d75d8e962e5c4c6583430bf34ba03210 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:02.858481 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f1e3c-834a-4ef0-92c4-6e667d8edbf0 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:02.858770 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f1e3c-834a-4ef0-92c4-6e667d8edbf0 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:02.859622 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f1e3c-834a-4ef0-92c4-6e667d8edbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:02.872413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d5fb5-16c5-4bac-ac57-9a77645056ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:02.873280 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d5fb5-16c5-4bac-ac57-9a77645056ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:02.873648 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d5fb5-16c5-4bac-ac57-9a77645056ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:02.874161 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d5fb5-16c5-4bac-ac57-9a77645056ed admin admin] PATH_INFO: `/v3/users/590c10c9351748deac98608e04380b4c` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:02.876001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d5fb5-16c5-4bac-ac57-9a77645056ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=590c10c9351748deac98608e04380b4c)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:02.877941 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d5fb5-16c5-4bac-ac57-9a77645056ed admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:02.879010 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-325f1e3c-834a-4ef0-92c4-6e667d8edbf0 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:02.880258 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-325f1e3c-834a-4ef0-92c4-6e667d8edbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:02.896320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1000/1983] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:02.897857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2fbb2-9bda-49b4-96c5-bc1f1c43ccb6 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:02.898142 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2fbb2-9bda-49b4-96c5-bc1f1c43ccb6 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:02.898423 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2fbb2-9bda-49b4-96c5-bc1f1c43ccb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:02.914949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:02.923458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-1cd2fbb2-9bda-49b4-96c5-bc1f1c43ccb6 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:02.925114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-1cd2fbb2-9bda-49b4-96c5-bc1f1c43ccb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:02.939391 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1001/1984] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:02.942330 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2fbb2-9bda-49b4-96c5-bc1f1c43ccb6 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:02.944946 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1d1d5fb5-16c5-4bac-ac57-9a77645056ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '590c10c9351748deac98608e04380b4c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:02.951691 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2fbb2-9bda-49b4-96c5-bc1f1c43ccb6 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:02.956595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d72513-3c74-457d-87f5-b466da524504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:02.957252 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d72513-3c74-457d-87f5-b466da524504 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:02.957536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d72513-3c74-457d-87f5-b466da524504 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:02.957661 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1d1d5fb5-16c5-4bac-ac57-9a77645056ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '590c10c9351748deac98608e04380b4c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:02.957815 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d72513-3c74-457d-87f5-b466da524504 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:02.958537 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d72513-3c74-457d-87f5-b466da524504 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:46:02.958843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d72513-3c74-457d-87f5-b466da524504 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:02.960482 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d72513-3c74-457d-87f5-b466da524504 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:02.965200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1d1d5fb5-16c5-4bac-ac57-9a77645056ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '590c10c9351748deac98608e04380b4c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:02.965896 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1d1d5fb5-16c5-4bac-ac57-9a77645056ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '590c10c9351748deac98608e04380b4c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:02.972506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1002/1985] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:46:02.973403 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1d1d5fb5-16c5-4bac-ac57-9a77645056ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '590c10c9351748deac98608e04380b4c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:02.976716 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d72513-3c74-457d-87f5-b466da524504 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:02.978398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f1d72513-3c74-457d-87f5-b466da524504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:02.979604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 984/1986] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:02 2022] DELETE /identity/v3/users/590c10c9351748deac98608e04380b4c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:02.996418 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d72513-3c74-457d-87f5-b466da524504 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:03.054486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5185b1e-8035-47a2-a641-71f43cac7d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:03.055242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5185b1e-8035-47a2-a641-71f43cac7d62 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:03.055594 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5185b1e-8035-47a2-a641-71f43cac7d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:03.055942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5185b1e-8035-47a2-a641-71f43cac7d62 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:03.056742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5185b1e-8035-47a2-a641-71f43cac7d62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:03.058740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5185b1e-8035-47a2-a641-71f43cac7d62 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:03.086137 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1003/1987] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:02 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:03.090391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d5fb5-16c5-4bac-ac57-9a77645056ed admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:03.101381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d5fb5-16c5-4bac-ac57-9a77645056ed admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:03.164711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81863c88-7861-4fdc-a882-4797aa450edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:03.165401 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81863c88-7861-4fdc-a882-4797aa450edf admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:03.165688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81863c88-7861-4fdc-a882-4797aa450edf admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:03.165966 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81863c88-7861-4fdc-a882-4797aa450edf admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:03.166661 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81863c88-7861-4fdc-a882-4797aa450edf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:03.168652 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81863c88-7861-4fdc-a882-4797aa450edf admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:03.189840 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 985/1988] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:03 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:03.193981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5185b1e-8035-47a2-a641-71f43cac7d62 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:03.204433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5185b1e-8035-47a2-a641-71f43cac7d62 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:03.209048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e2773-4c62-4608-ac32-76830adf7b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:03.209713 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e2773-4c62-4608-ac32-76830adf7b49 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:03.210002 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e2773-4c62-4608-ac32-76830adf7b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:03.210281 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e2773-4c62-4608-ac32-76830adf7b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:03.211048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365e2773-4c62-4608-ac32-76830adf7b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:03.212708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365e2773-4c62-4608-ac32-76830adf7b49 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:03.226510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1004/1989] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:03.227985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81863c88-7861-4fdc-a882-4797aa450edf admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:03.238912 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81863c88-7861-4fdc-a882-4797aa450edf admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:03.244179 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d4a48-a6ec-4cf8-9690-d1409dce6f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:03.244937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d4a48-a6ec-4cf8-9690-d1409dce6f92 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:03.245223 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d4a48-a6ec-4cf8-9690-d1409dce6f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:03.245496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d4a48-a6ec-4cf8-9690-d1409dce6f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:03.246282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d4a48-a6ec-4cf8-9690-d1409dce6f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:03.247908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d4a48-a6ec-4cf8-9690-d1409dce6f92 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:03.260544 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e2773-4c62-4608-ac32-76830adf7b49 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:03.262899 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-365e2773-4c62-4608-ac32-76830adf7b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:03.268582 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 986/1990] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:03.269938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d4a48-a6ec-4cf8-9690-d1409dce6f92 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:03.278964 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e2773-4c62-4608-ac32-76830adf7b49 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:03.281965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d4a48-a6ec-4cf8-9690-d1409dce6f92 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:03.287013 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7fc43-e56b-483e-93b1-05f33f75e0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:03.287741 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7fc43-e56b-483e-93b1-05f33f75e0ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:03.288029 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7fc43-e56b-483e-93b1-05f33f75e0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:03.288315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f232324-3362-4a86-9051-29feb20b86c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:03.288432 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7fc43-e56b-483e-93b1-05f33f75e0ae admin admin] PATH_INFO: `/v3/projects/2aae088dfa144636bcfa79d0643a6f19` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:03.289727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f232324-3362-4a86-9051-29feb20b86c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:03.289844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7fc43-e56b-483e-93b1-05f33f75e0ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aae088dfa144636bcfa79d0643a6f19)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:03.290091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f232324-3362-4a86-9051-29feb20b86c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:03.290491 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f232324-3362-4a86-9051-29feb20b86c1 admin admin] PATH_INFO: `/v3/projects/97d7e536c4254f469c6da669481eb8e6/users/72365998f1bb459cbff28ca0b267fe69/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:03.292485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7fc43-e56b-483e-93b1-05f33f75e0ae admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:03.308871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f232324-3362-4a86-9051-29feb20b86c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97d7e536c4254f469c6da669481eb8e6, user_id=72365998f1bb459cbff28ca0b267fe69, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:03.311699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f232324-3362-4a86-9051-29feb20b86c1 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:03.325309 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 987/1991] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:03 2022] PUT /identity/v3/projects/97d7e536c4254f469c6da669481eb8e6/users/72365998f1bb459cbff28ca0b267fe69/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:03.332740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b38a48-e20e-434f-b1e7-2544a9aae4f6 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:03.333111 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b38a48-e20e-434f-b1e7-2544a9aae4f6 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:03.333646 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b38a48-e20e-434f-b1e7-2544a9aae4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:03.348507 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-91c7fc43-e56b-483e-93b1-05f33f75e0ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aae088dfa144636bcfa79d0643a6f19', 'request_id': 'req-91c7fc43-e56b-483e-93b1-05f33f75e0ae'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:03.351192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:03.359045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-53b38a48-e20e-434f-b1e7-2544a9aae4f6 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:03.360379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-53b38a48-e20e-434f-b1e7-2544a9aae4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:03.361813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-91c7fc43-e56b-483e-93b1-05f33f75e0ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aae088dfa144636bcfa79d0643a6f19', 'request_id': 'req-91c7fc43-e56b-483e-93b1-05f33f75e0ae'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:03.381321 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-91c7fc43-e56b-483e-93b1-05f33f75e0ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aae088dfa144636bcfa79d0643a6f19', 'request_id': 'req-91c7fc43-e56b-483e-93b1-05f33f75e0ae'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:03.397970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-91c7fc43-e56b-483e-93b1-05f33f75e0ae admin admin] The token cache is being invalidate because project 2aae088dfa144636bcfa79d0643a6f19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:03.398268 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-91c7fc43-e56b-483e-93b1-05f33f75e0ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:03.399012 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-91c7fc43-e56b-483e-93b1-05f33f75e0ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:03.402063 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1005/1992] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:03 2022] DELETE /identity/v3/projects/2aae088dfa144636bcfa79d0643a6f19 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:03.406710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055c03c-d681-4e3d-ba02-739c1f1a11e4 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:03.406997 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055c03c-d681-4e3d-ba02-739c1f1a11e4 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:03.407273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055c03c-d681-4e3d-ba02-739c1f1a11e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:03.429976 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-0055c03c-d681-4e3d-ba02-739c1f1a11e4 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:03.431260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-0055c03c-d681-4e3d-ba02-739c1f1a11e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:03.443968 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:03.455577 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 988/1993] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:03.465835 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b38a48-e20e-434f-b1e7-2544a9aae4f6 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:03.467272 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-53b38a48-e20e-434f-b1e7-2544a9aae4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:03.527887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1006/1994] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:03.529555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10e61d-2676-4403-b8e9-00c767d7f425 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:03.529852 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10e61d-2676-4403-b8e9-00c767d7f425 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:03.530135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10e61d-2676-4403-b8e9-00c767d7f425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:03.537299 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b38a48-e20e-434f-b1e7-2544a9aae4f6 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:03.542949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ac2ac-e478-4462-a170-cee09d203b8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:03.543864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ac2ac-e478-4462-a170-cee09d203b8a service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:03.544820 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ac2ac-e478-4462-a170-cee09d203b8a service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:03.545286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ac2ac-e478-4462-a170-cee09d203b8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:03.550555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-ef10e61d-2676-4403-b8e9-00c767d7f425 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:03.551802 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ef10e61d-2676-4403-b8e9-00c767d7f425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:03.561730 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1007/1995] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:03.565641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10e61d-2676-4403-b8e9-00c767d7f425 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:03.568087 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ac2ac-e478-4462-a170-cee09d203b8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:03.570527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ac2ac-e478-4462-a170-cee09d203b8a service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:03.574887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10e61d-2676-4403-b8e9-00c767d7f425 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:03.580456 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d35052-10f0-4bb0-b734-dd8189b26fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:03.581312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d35052-10f0-4bb0-b734-dd8189b26fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:03.581732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d35052-10f0-4bb0-b734-dd8189b26fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:03.582102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d35052-10f0-4bb0-b734-dd8189b26fe3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:03.583019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d35052-10f0-4bb0-b734-dd8189b26fe3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:46:03.583491 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d35052-10f0-4bb0-b734-dd8189b26fe3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:03.585206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d35052-10f0-4bb0-b734-dd8189b26fe3 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:03.597904 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 989/1996] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:46:03.607737 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1008/1997] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:46:03.608115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ac2ac-e478-4462-a170-cee09d203b8a service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:03.622326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ac2ac-e478-4462-a170-cee09d203b8a service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:03.627647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a41da-3dad-4a4c-941c-07333d4d7e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:03.629273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a41da-3dad-4a4c-941c-07333d4d7e69 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:03.629273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a41da-3dad-4a4c-941c-07333d4d7e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:03.629273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a41da-3dad-4a4c-941c-07333d4d7e69 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:03.629881 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a41da-3dad-4a4c-941c-07333d4d7e69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:03.632025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a41da-3dad-4a4c-941c-07333d4d7e69 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:03.660636 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 990/1998] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:03 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:46:03.669670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d35052-10f0-4bb0-b734-dd8189b26fe3 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:03.677789 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d35052-10f0-4bb0-b734-dd8189b26fe3 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:03.772064 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6921e58-e617-4853-a76e-1dc72e7d2593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:03.772990 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6921e58-e617-4853-a76e-1dc72e7d2593 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:03.773417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6921e58-e617-4853-a76e-1dc72e7d2593 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:03.773924 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6921e58-e617-4853-a76e-1dc72e7d2593 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:03.774722 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6921e58-e617-4853-a76e-1dc72e7d2593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:03.777007 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6921e58-e617-4853-a76e-1dc72e7d2593 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:03.809574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1009/1999] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:03 2022] POST /identity/v3/users => generated 295 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:03.814146 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a41da-3dad-4a4c-941c-07333d4d7e69 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:03.828371 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a41da-3dad-4a4c-941c-07333d4d7e69 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:03.840010 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ee452-cd70-4e7d-b2fa-5017ddfbe705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:03.845831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ee452-cd70-4e7d-b2fa-5017ddfbe705 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:03.846210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ee452-cd70-4e7d-b2fa-5017ddfbe705 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:03.846562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ee452-cd70-4e7d-b2fa-5017ddfbe705 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:03.847406 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ee452-cd70-4e7d-b2fa-5017ddfbe705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:03.866439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ee452-cd70-4e7d-b2fa-5017ddfbe705 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:03.895648 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 991/2000] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:03.901596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6921e58-e617-4853-a76e-1dc72e7d2593 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:03.924824 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6921e58-e617-4853-a76e-1dc72e7d2593 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:03.937385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cf83b-c3f2-439b-a17a-d6f0d71bf85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:03.947682 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cf83b-c3f2-439b-a17a-d6f0d71bf85f admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:03.947983 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cf83b-c3f2-439b-a17a-d6f0d71bf85f admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:03.948269 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cf83b-c3f2-439b-a17a-d6f0d71bf85f admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:03.949270 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cf83b-c3f2-439b-a17a-d6f0d71bf85f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:03.950804 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cf83b-c3f2-439b-a17a-d6f0d71bf85f admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:03.969136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1010/2001] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:03.973777 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ee452-cd70-4e7d-b2fa-5017ddfbe705 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:04.005814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ee452-cd70-4e7d-b2fa-5017ddfbe705 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:04.013217 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db01e6-a7c7-4583-b3b5-aee087981460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:04.014338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db01e6-a7c7-4583-b3b5-aee087981460 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:04.014625 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db01e6-a7c7-4583-b3b5-aee087981460 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:04.014911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db01e6-a7c7-4583-b3b5-aee087981460 admin admin] PATH_INFO: `/v3/projects/80b49661e3e44843b5d09045017f53bb/users/bcc7712501ba45ef80c52dcdd4258fe1/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:04.036176 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db01e6-a7c7-4583-b3b5-aee087981460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80b49661e3e44843b5d09045017f53bb, user_id=bcc7712501ba45ef80c52dcdd4258fe1, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:04.039329 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db01e6-a7c7-4583-b3b5-aee087981460 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:04.059050 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 992/2002] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:03 2022] PUT /identity/v3/projects/80b49661e3e44843b5d09045017f53bb/users/bcc7712501ba45ef80c52dcdd4258fe1/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:04.070109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd6522-e8d8-4eed-90f9-ef2f17abaa37 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:04.070768 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd6522-e8d8-4eed-90f9-ef2f17abaa37 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:04.071116 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd6522-e8d8-4eed-90f9-ef2f17abaa37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:04.107199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-abdd6522-e8d8-4eed-90f9-ef2f17abaa37 None None] MFA Rules not processed for user `bcc7712501ba45ef80c52dcdd4258fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:04.108450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-abdd6522-e8d8-4eed-90f9-ef2f17abaa37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:04.148445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:04.218789 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1011/2003] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:04.223030 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db01e6-a7c7-4583-b3b5-aee087981460 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:04.305518 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db01e6-a7c7-4583-b3b5-aee087981460 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:04.310553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd50439-3662-4524-8e81-939e7c0c8b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:04.311084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd50439-3662-4524-8e81-939e7c0c8b15 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:04.311292 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd50439-3662-4524-8e81-939e7c0c8b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:04.311617 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd50439-3662-4524-8e81-939e7c0c8b15 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:04.312890 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd50439-3662-4524-8e81-939e7c0c8b15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:04.314802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd50439-3662-4524-8e81-939e7c0c8b15 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:04.344289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 993/2004] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:04 2022] POST /identity/v3/projects => generated 389 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:46:04.346246 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd6522-e8d8-4eed-90f9-ef2f17abaa37 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:04.359831 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd6522-e8d8-4eed-90f9-ef2f17abaa37 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:04.414409 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e911f-3973-487e-b8d7-b80f4381cef1 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:04.415436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e911f-3973-487e-b8d7-b80f4381cef1 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:04.415831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e911f-3973-487e-b8d7-b80f4381cef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:04.438112 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd478bd6-309a-447d-81d2-bda4a622e187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:04.439096 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd478bd6-309a-447d-81d2-bda4a622e187 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:04.439424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd478bd6-309a-447d-81d2-bda4a622e187 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:04.439706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd478bd6-309a-447d-81d2-bda4a622e187 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:04.440412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd478bd6-309a-447d-81d2-bda4a622e187 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:04.442464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd478bd6-309a-447d-81d2-bda4a622e187 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:04.445978 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-2c4e911f-3973-487e-b8d7-b80f4381cef1 None None] MFA Rules not processed for user `72365998f1bb459cbff28ca0b267fe69`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:04.447464 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2c4e911f-3973-487e-b8d7-b80f4381cef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:04.466837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1012/2005] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:04 2022] POST /identity/v3/users => generated 301 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:46:04.468476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd478bd6-309a-447d-81d2-bda4a622e187 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:04.469626 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:04.480513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd478bd6-309a-447d-81d2-bda4a622e187 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:04.486866 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15576760-529c-469c-a2a7-677b9327e37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:04.487561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15576760-529c-469c-a2a7-677b9327e37d admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:04.487850 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15576760-529c-469c-a2a7-677b9327e37d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:04.488127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15576760-529c-469c-a2a7-677b9327e37d admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:04.488867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15576760-529c-469c-a2a7-677b9327e37d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:04.490591 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15576760-529c-469c-a2a7-677b9327e37d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:04.503408 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1013/2006] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:46:04.506798 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15576760-529c-469c-a2a7-677b9327e37d admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:04.520092 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15576760-529c-469c-a2a7-677b9327e37d admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:04.524926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8682fa3-8350-4944-96ea-6a7daa744aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:04.525892 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8682fa3-8350-4944-96ea-6a7daa744aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:04.526418 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8682fa3-8350-4944-96ea-6a7daa744aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:04.526736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8682fa3-8350-4944-96ea-6a7daa744aa7 admin admin] PATH_INFO: `/v3/projects/1f0331c96fcf42e495c3b449d786ef67/users/72a357d2808343e790c7949c1b943051/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:04.529590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 994/2007] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:04.540944 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e911f-3973-487e-b8d7-b80f4381cef1 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:04.544366 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2c4e911f-3973-487e-b8d7-b80f4381cef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:04.552135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8682fa3-8350-4944-96ea-6a7daa744aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f0331c96fcf42e495c3b449d786ef67, user_id=72a357d2808343e790c7949c1b943051, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:04.554826 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8682fa3-8350-4944-96ea-6a7daa744aa7 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:04.569327 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1014/2008] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:04 2022] PUT /identity/v3/projects/1f0331c96fcf42e495c3b449d786ef67/users/72a357d2808343e790c7949c1b943051/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:04.573264 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8682fa3-8350-4944-96ea-6a7daa744aa7 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:04.627696 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e911f-3973-487e-b8d7-b80f4381cef1 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:04.656526 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8682fa3-8350-4944-96ea-6a7daa744aa7 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:04.661738 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301dba75-db60-4ae3-82c9-49ea1ca43fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:04.662487 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301dba75-db60-4ae3-82c9-49ea1ca43fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:04.662964 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301dba75-db60-4ae3-82c9-49ea1ca43fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:04.663777 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301dba75-db60-4ae3-82c9-49ea1ca43fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:04.664231 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301dba75-db60-4ae3-82c9-49ea1ca43fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:04.666080 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301dba75-db60-4ae3-82c9-49ea1ca43fe1 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:04.689515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1015/2009] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:46:04.690587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301dba75-db60-4ae3-82c9-49ea1ca43fe1 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:04.695472 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45a87e-13a9-4ef2-bb36-ac7a1912e7cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:04.696555 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45a87e-13a9-4ef2-bb36-ac7a1912e7cf service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:04.697232 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45a87e-13a9-4ef2-bb36-ac7a1912e7cf service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:04.697664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d45a87e-13a9-4ef2-bb36-ac7a1912e7cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:04.702699 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301dba75-db60-4ae3-82c9-49ea1ca43fe1 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:04.708650 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109fc9e-ec00-41f7-82e3-bdd14b4d2241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:04.710557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109fc9e-ec00-41f7-82e3-bdd14b4d2241 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:04.710557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109fc9e-ec00-41f7-82e3-bdd14b4d2241 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:04.710557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109fc9e-ec00-41f7-82e3-bdd14b4d2241 admin admin] PATH_INFO: `/v3/projects/1f0331c96fcf42e495c3b449d786ef67/users/72a357d2808343e790c7949c1b943051/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:04.712581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109fc9e-ec00-41f7-82e3-bdd14b4d2241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f0331c96fcf42e495c3b449d786ef67, user_id=72a357d2808343e790c7949c1b943051, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:04.715216 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109fc9e-ec00-41f7-82e3-bdd14b4d2241 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:04.739952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1016/2010] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:04 2022] PUT /identity/v3/projects/1f0331c96fcf42e495c3b449d786ef67/users/72a357d2808343e790c7949c1b943051/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:04.747188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe7ae9-3aec-40fe-9fdd-76a10b2a44aa None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:04.747525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe7ae9-3aec-40fe-9fdd-76a10b2a44aa None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:04.747811 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe7ae9-3aec-40fe-9fdd-76a10b2a44aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:04.763699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45a87e-13a9-4ef2-bb36-ac7a1912e7cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:04.765351 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d45a87e-13a9-4ef2-bb36-ac7a1912e7cf service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:04.765772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:04.777742 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 995/2011] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:46:04.785007 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-3abe7ae9-3aec-40fe-9fdd-76a10b2a44aa None None] MFA Rules not processed for user `72a357d2808343e790c7949c1b943051`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:04.786225 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-3abe7ae9-3aec-40fe-9fdd-76a10b2a44aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:04.874960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1017/2012] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:04.883036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45a87e-13a9-4ef2-bb36-ac7a1912e7cf service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:04.888973 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2d45a87e-13a9-4ef2-bb36-ac7a1912e7cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:04.901471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d45a87e-13a9-4ef2-bb36-ac7a1912e7cf service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:04.940311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:04.968993 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64b648-d358-4fad-8ecb-8dfae47beb70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:04.970006 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64b648-d358-4fad-8ecb-8dfae47beb70 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:04.970285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64b648-d358-4fad-8ecb-8dfae47beb70 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:04.970644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64b648-d358-4fad-8ecb-8dfae47beb70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:04.982383 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64b648-d358-4fad-8ecb-8dfae47beb70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:04.984328 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64b648-d358-4fad-8ecb-8dfae47beb70 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:04.996744 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 996/2013] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:05.187720 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe7ae9-3aec-40fe-9fdd-76a10b2a44aa None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:05.257927 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe7ae9-3aec-40fe-9fdd-76a10b2a44aa None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:05.269450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08224f41-d7e3-4416-8da8-675d9378036a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:05.269450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08224f41-d7e3-4416-8da8-675d9378036a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:05.269450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08224f41-d7e3-4416-8da8-675d9378036a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:05.269450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08224f41-d7e3-4416-8da8-675d9378036a admin admin] PATH_INFO: `/v3/users/bcc7712501ba45ef80c52dcdd4258fe1` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:05.269946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08224f41-d7e3-4416-8da8-675d9378036a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcc7712501ba45ef80c52dcdd4258fe1)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:05.270234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08224f41-d7e3-4416-8da8-675d9378036a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:05.321683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-08224f41-d7e3-4416-8da8-675d9378036a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcc7712501ba45ef80c52dcdd4258fe1'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:05.334335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-08224f41-d7e3-4416-8da8-675d9378036a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcc7712501ba45ef80c52dcdd4258fe1'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:05.340752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-08224f41-d7e3-4416-8da8-675d9378036a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcc7712501ba45ef80c52dcdd4258fe1'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:05.347690 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-08224f41-d7e3-4416-8da8-675d9378036a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcc7712501ba45ef80c52dcdd4258fe1'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:05.348312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-08224f41-d7e3-4416-8da8-675d9378036a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcc7712501ba45ef80c52dcdd4258fe1'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:05.354065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1018/2014] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:05 2022] DELETE /identity/v3/users/bcc7712501ba45ef80c52dcdd4258fe1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:05.358501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c64b4-8968-493b-ae4c-4f35525df87e None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:05.358811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c64b4-8968-493b-ae4c-4f35525df87e None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:05.359097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c64b4-8968-493b-ae4c-4f35525df87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:05.376864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-707c64b4-8968-493b-ae4c-4f35525df87e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:05.377986 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-707c64b4-8968-493b-ae4c-4f35525df87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:05.437706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 997/2015] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:05.447307 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08224f41-d7e3-4416-8da8-675d9378036a admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:05.448736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-08224f41-d7e3-4416-8da8-675d9378036a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:05.462995 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:05.508048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08224f41-d7e3-4416-8da8-675d9378036a admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:05.511687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782de2d-b03b-446b-b89f-841f413c311a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:05.512343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782de2d-b03b-446b-b89f-841f413c311a service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:05.512623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782de2d-b03b-446b-b89f-841f413c311a service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:05.512918 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782de2d-b03b-446b-b89f-841f413c311a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:05.523243 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782de2d-b03b-446b-b89f-841f413c311a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:05.524700 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782de2d-b03b-446b-b89f-841f413c311a service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:05.534489 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1019/2016] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:05.797174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c64b4-8968-493b-ae4c-4f35525df87e None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:05.798873 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-707c64b4-8968-493b-ae4c-4f35525df87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:05.811879 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c64b4-8968-493b-ae4c-4f35525df87e None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:05.819966 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b1eeb-664a-41b9-9118-a249819aa35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:05.820853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b1eeb-664a-41b9-9118-a249819aa35b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:05.821128 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b1eeb-664a-41b9-9118-a249819aa35b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:05.821638 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b1eeb-664a-41b9-9118-a249819aa35b admin admin] PATH_INFO: `/v3/projects/80b49661e3e44843b5d09045017f53bb` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:05.823307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b1eeb-664a-41b9-9118-a249819aa35b admin admin] RBAC: Authorizing `identity:delete_project(project_id=80b49661e3e44843b5d09045017f53bb)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:05.825607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b1eeb-664a-41b9-9118-a249819aa35b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:05.882534 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-294b1eeb-664a-41b9-9118-a249819aa35b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80b49661e3e44843b5d09045017f53bb', 'request_id': 'req-294b1eeb-664a-41b9-9118-a249819aa35b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:05.893126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-294b1eeb-664a-41b9-9118-a249819aa35b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80b49661e3e44843b5d09045017f53bb', 'request_id': 'req-294b1eeb-664a-41b9-9118-a249819aa35b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:05.901671 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-294b1eeb-664a-41b9-9118-a249819aa35b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80b49661e3e44843b5d09045017f53bb', 'request_id': 'req-294b1eeb-664a-41b9-9118-a249819aa35b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:05.916388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-294b1eeb-664a-41b9-9118-a249819aa35b admin admin] The token cache is being invalidate because project 80b49661e3e44843b5d09045017f53bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:05.916591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-294b1eeb-664a-41b9-9118-a249819aa35b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:05.917259 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-294b1eeb-664a-41b9-9118-a249819aa35b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:05.918956 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 998/2017] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:05 2022] DELETE /identity/v3/projects/80b49661e3e44843b5d09045017f53bb => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:05.923845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782de2d-b03b-446b-b89f-841f413c311a service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:05.937637 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782de2d-b03b-446b-b89f-841f413c311a service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:05.995898 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c446045-381f-470a-8d18-b39ecb791333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:05.996781 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c446045-381f-470a-8d18-b39ecb791333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:05.997074 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c446045-381f-470a-8d18-b39ecb791333 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:05.997325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c446045-381f-470a-8d18-b39ecb791333 admin admin] PATH_INFO: `/v3/users/72a357d2808343e790c7949c1b943051` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:05.998520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c446045-381f-470a-8d18-b39ecb791333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72a357d2808343e790c7949c1b943051)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:06.001360 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c446045-381f-470a-8d18-b39ecb791333 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:06.053930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8c446045-381f-470a-8d18-b39ecb791333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72a357d2808343e790c7949c1b943051'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:06.066201 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8c446045-381f-470a-8d18-b39ecb791333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72a357d2808343e790c7949c1b943051'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:06.074254 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8c446045-381f-470a-8d18-b39ecb791333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72a357d2808343e790c7949c1b943051'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:06.081907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8c446045-381f-470a-8d18-b39ecb791333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72a357d2808343e790c7949c1b943051'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:06.082582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8c446045-381f-470a-8d18-b39ecb791333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72a357d2808343e790c7949c1b943051'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:06.093749 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1020/2018] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:05 2022] DELETE /identity/v3/users/72a357d2808343e790c7949c1b943051 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:06.411341 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b1eeb-664a-41b9-9118-a249819aa35b admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:06.414227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-294b1eeb-664a-41b9-9118-a249819aa35b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:06.429510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b1eeb-664a-41b9-9118-a249819aa35b admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:06.487990 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:06.495681 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564147e-ed23-4c6c-bfa2-44ec253f9936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:06.496691 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564147e-ed23-4c6c-bfa2-44ec253f9936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:06.497469 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564147e-ed23-4c6c-bfa2-44ec253f9936 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:06.497773 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564147e-ed23-4c6c-bfa2-44ec253f9936 admin admin] PATH_INFO: `/v3/projects/1f0331c96fcf42e495c3b449d786ef67` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:06.499086 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564147e-ed23-4c6c-bfa2-44ec253f9936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f0331c96fcf42e495c3b449d786ef67)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:06.501821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564147e-ed23-4c6c-bfa2-44ec253f9936 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:06.553391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-7564147e-ed23-4c6c-bfa2-44ec253f9936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f0331c96fcf42e495c3b449d786ef67', 'request_id': 'req-7564147e-ed23-4c6c-bfa2-44ec253f9936'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:06.561868 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-7564147e-ed23-4c6c-bfa2-44ec253f9936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f0331c96fcf42e495c3b449d786ef67', 'request_id': 'req-7564147e-ed23-4c6c-bfa2-44ec253f9936'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:06.568389 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-7564147e-ed23-4c6c-bfa2-44ec253f9936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f0331c96fcf42e495c3b449d786ef67', 'request_id': 'req-7564147e-ed23-4c6c-bfa2-44ec253f9936'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:06.580085 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-7564147e-ed23-4c6c-bfa2-44ec253f9936 admin admin] The token cache is being invalidate because project 1f0331c96fcf42e495c3b449d786ef67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:06.580423 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-7564147e-ed23-4c6c-bfa2-44ec253f9936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:06.581141 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-7564147e-ed23-4c6c-bfa2-44ec253f9936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:06.583198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 999/2019] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:06 2022] DELETE /identity/v3/projects/1f0331c96fcf42e495c3b449d786ef67 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:06.588704 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28128262-a6ba-45fa-9981-6bb4bf8a522c None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:06.589198 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28128262-a6ba-45fa-9981-6bb4bf8a522c None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:06.589710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28128262-a6ba-45fa-9981-6bb4bf8a522c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:06.608667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-28128262-a6ba-45fa-9981-6bb4bf8a522c None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:06.609940 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-28128262-a6ba-45fa-9981-6bb4bf8a522c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:06.639746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:06.673103 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1021/2020] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:06.680881 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c9eb7-5bf0-48a8-973e-663eb3940e6f None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:06.681164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c9eb7-5bf0-48a8-973e-663eb3940e6f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:06.681443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c9eb7-5bf0-48a8-973e-663eb3940e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:06.700774 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c13c9eb7-5bf0-48a8-973e-663eb3940e6f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:06.702000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c13c9eb7-5bf0-48a8-973e-663eb3940e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:06.714276 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1000/2021] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:06.720622 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28128262-a6ba-45fa-9981-6bb4bf8a522c None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:06.730866 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28128262-a6ba-45fa-9981-6bb4bf8a522c None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:06.736658 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b67d55-9de8-436c-ab9c-f613f3318d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:06.737422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b67d55-9de8-436c-ab9c-f613f3318d94 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:06.737709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b67d55-9de8-436c-ab9c-f613f3318d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:06.737989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b67d55-9de8-436c-ab9c-f613f3318d94 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:06.738967 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b67d55-9de8-436c-ab9c-f613f3318d94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:46:06.740017 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b67d55-9de8-436c-ab9c-f613f3318d94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:06.741741 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b67d55-9de8-436c-ab9c-f613f3318d94 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:06.759507 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1022/2022] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:46:06.761157 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c9eb7-5bf0-48a8-973e-663eb3940e6f None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:06.774179 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c9eb7-5bf0-48a8-973e-663eb3940e6f None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:06.780452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9d4e2-32ed-4ce2-80c9-68a758849337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:06.781700 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9d4e2-32ed-4ce2-80c9-68a758849337 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:06.781989 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9d4e2-32ed-4ce2-80c9-68a758849337 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:06.782267 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9d4e2-32ed-4ce2-80c9-68a758849337 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:06.782990 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9d4e2-32ed-4ce2-80c9-68a758849337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:06.785538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9d4e2-32ed-4ce2-80c9-68a758849337 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:06.825168 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1001/2023] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:06 2022] POST /identity/v3/projects => generated 379 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:06.829964 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b67d55-9de8-436c-ab9c-f613f3318d94 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:06.841524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b67d55-9de8-436c-ab9c-f613f3318d94 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:06.915231 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7a6a0-b331-4492-bee7-1bcdb210159e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:06.916109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7a6a0-b331-4492-bee7-1bcdb210159e admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:06.916504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7a6a0-b331-4492-bee7-1bcdb210159e admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:06.916889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7a6a0-b331-4492-bee7-1bcdb210159e admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:06.917698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7a6a0-b331-4492-bee7-1bcdb210159e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:06.924070 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7a6a0-b331-4492-bee7-1bcdb210159e admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:06.955526 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1023/2024] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:06 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:46:06.956245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9d4e2-32ed-4ce2-80c9-68a758849337 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:06.967791 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9d4e2-32ed-4ce2-80c9-68a758849337 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:06.972735 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c5352-0776-4cdd-9a7c-98485c287aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:06.973521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c5352-0776-4cdd-9a7c-98485c287aea admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:06.974338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c5352-0776-4cdd-9a7c-98485c287aea admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:06.974742 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c5352-0776-4cdd-9a7c-98485c287aea admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:06.975787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c5352-0776-4cdd-9a7c-98485c287aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:06.977574 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c5352-0776-4cdd-9a7c-98485c287aea admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:06.990801 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1002/2025] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:46:06.995054 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7a6a0-b331-4492-bee7-1bcdb210159e admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:07.007478 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7a6a0-b331-4492-bee7-1bcdb210159e admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:07.013535 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfc6d9-09b0-43ca-850e-7c5c32dd4cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:07.014311 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfc6d9-09b0-43ca-850e-7c5c32dd4cbd admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:07.014692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfc6d9-09b0-43ca-850e-7c5c32dd4cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:07.015120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfc6d9-09b0-43ca-850e-7c5c32dd4cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:07.016462 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfc6d9-09b0-43ca-850e-7c5c32dd4cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:07.018717 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfc6d9-09b0-43ca-850e-7c5c32dd4cbd admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:07.032251 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1024/2026] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:07.036223 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c5352-0776-4cdd-9a7c-98485c287aea admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:07.055433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c5352-0776-4cdd-9a7c-98485c287aea admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:07.060635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442bf0bc-295b-49df-8e68-93390dbe3090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:07.061706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bf0bc-295b-49df-8e68-93390dbe3090 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:07.062531 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bf0bc-295b-49df-8e68-93390dbe3090 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:07.062810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442bf0bc-295b-49df-8e68-93390dbe3090 admin admin] PATH_INFO: `/v3/projects/aed58fc9ac474ae39b24a9ef634fa6a5/users/62f33d76d51a4cc9a9f97f4ae2f5b0fe/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:07.082534 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442bf0bc-295b-49df-8e68-93390dbe3090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aed58fc9ac474ae39b24a9ef634fa6a5, user_id=62f33d76d51a4cc9a9f97f4ae2f5b0fe, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:07.085955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442bf0bc-295b-49df-8e68-93390dbe3090 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:07.107244 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1003/2027] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:07 2022] PUT /identity/v3/projects/aed58fc9ac474ae39b24a9ef634fa6a5/users/62f33d76d51a4cc9a9f97f4ae2f5b0fe/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:07.118121 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928b92e-3415-4833-b06d-b9ecdfe123b0 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:07.118411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928b92e-3415-4833-b06d-b9ecdfe123b0 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:07.118691 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928b92e-3415-4833-b06d-b9ecdfe123b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:07.154907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-1928b92e-3415-4833-b06d-b9ecdfe123b0 None None] MFA Rules not processed for user `62f33d76d51a4cc9a9f97f4ae2f5b0fe`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:07.156401 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-1928b92e-3415-4833-b06d-b9ecdfe123b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:07.286510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1025/2028] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:07.286872 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442bf0bc-295b-49df-8e68-93390dbe3090 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:07.390105 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78197957-bc09-4444-bc86-30b34471960a None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:07.390402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78197957-bc09-4444-bc86-30b34471960a None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:07.390779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78197957-bc09-4444-bc86-30b34471960a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:07.393068 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442bf0bc-295b-49df-8e68-93390dbe3090 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:07.403570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e4003-0550-4dee-9720-111301ccf9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:07.404896 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e4003-0550-4dee-9720-111301ccf9f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:07.406488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e4003-0550-4dee-9720-111301ccf9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:07.406862 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9e4003-0550-4dee-9720-111301ccf9f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:07.407684 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e4003-0550-4dee-9720-111301ccf9f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:07.417771 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9e4003-0550-4dee-9720-111301ccf9f5 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:07.419875 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-78197957-bc09-4444-bc86-30b34471960a None None] MFA Rules not processed for user `72365998f1bb459cbff28ca0b267fe69`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:07.422113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-78197957-bc09-4444-bc86-30b34471960a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:07.470028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:07.488438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1004/2029] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:07 2022] POST /identity/v3/projects => generated 379 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:46:07.488438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e4003-0550-4dee-9720-111301ccf9f5 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:07.503751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:07.535396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1026/2030] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:07.624014 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9e4003-0550-4dee-9720-111301ccf9f5 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:07.629810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ce468-4d39-44cb-bd3a-83740514971a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:07.630497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ce468-4d39-44cb-bd3a-83740514971a admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:07.630780 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ce468-4d39-44cb-bd3a-83740514971a admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:07.631057 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ce468-4d39-44cb-bd3a-83740514971a admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:07.631792 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ce468-4d39-44cb-bd3a-83740514971a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:07.633713 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ce468-4d39-44cb-bd3a-83740514971a admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:07.660339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78197957-bc09-4444-bc86-30b34471960a None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:07.661557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1005/2031] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:07 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:07.672361 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78197957-bc09-4444-bc86-30b34471960a None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:07.677402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c73438-79b5-4d15-acd3-aa9480b0b748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:07.678162 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c73438-79b5-4d15-acd3-aa9480b0b748 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:07.678519 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c73438-79b5-4d15-acd3-aa9480b0b748 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:07.678869 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c73438-79b5-4d15-acd3-aa9480b0b748 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:07.679746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c73438-79b5-4d15-acd3-aa9480b0b748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:07.681636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c73438-79b5-4d15-acd3-aa9480b0b748 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:07.694061 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1027/2032] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:07.698368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ce468-4d39-44cb-bd3a-83740514971a admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:07.709669 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ce468-4d39-44cb-bd3a-83740514971a admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:07.714703 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0aff4b-f36e-4198-8cb5-6631bf0d3618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:07.715415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0aff4b-f36e-4198-8cb5-6631bf0d3618 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:07.715705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0aff4b-f36e-4198-8cb5-6631bf0d3618 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:07.715989 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0aff4b-f36e-4198-8cb5-6631bf0d3618 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:07.716732 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0aff4b-f36e-4198-8cb5-6631bf0d3618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:07.718411 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0aff4b-f36e-4198-8cb5-6631bf0d3618 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:07.733240 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1006/2033] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:07.738310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c73438-79b5-4d15-acd3-aa9480b0b748 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:07.755916 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c73438-79b5-4d15-acd3-aa9480b0b748 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:07.771059 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c833ba-e224-46c4-b722-eb6844f23de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:07.772567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c833ba-e224-46c4-b722-eb6844f23de6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:07.774368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c833ba-e224-46c4-b722-eb6844f23de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:07.774368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c833ba-e224-46c4-b722-eb6844f23de6 admin admin] PATH_INFO: `/v3/projects/07fce36833d747e6b56108c85daf4143/users/7a3e43ed896a498fb9c6034e26075cac/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:07.801506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c833ba-e224-46c4-b722-eb6844f23de6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07fce36833d747e6b56108c85daf4143, user_id=7a3e43ed896a498fb9c6034e26075cac, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:07.806139 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c833ba-e224-46c4-b722-eb6844f23de6 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:07.841270 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1028/2034] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:07 2022] PUT /identity/v3/projects/07fce36833d747e6b56108c85daf4143/users/7a3e43ed896a498fb9c6034e26075cac/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:07.843150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a177c0-5050-4228-a529-4a395848204f None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:07.843504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a177c0-5050-4228-a529-4a395848204f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:07.844387 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a177c0-5050-4228-a529-4a395848204f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:07.881597 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-a5a177c0-5050-4228-a529-4a395848204f None None] MFA Rules not processed for user `7a3e43ed896a498fb9c6034e26075cac`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:07.882957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a5a177c0-5050-4228-a529-4a395848204f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:08.011315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1007/2035] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:08.016358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c833ba-e224-46c4-b722-eb6844f23de6 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:08.168422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c833ba-e224-46c4-b722-eb6844f23de6 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:08.182176 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bac75-1e1a-45c7-934b-d6e9ed6e2471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:08.183061 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bac75-1e1a-45c7-934b-d6e9ed6e2471 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:08.183342 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bac75-1e1a-45c7-934b-d6e9ed6e2471 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:08.183847 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bac75-1e1a-45c7-934b-d6e9ed6e2471 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:08.184563 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836bac75-1e1a-45c7-934b-d6e9ed6e2471 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:08.186496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836bac75-1e1a-45c7-934b-d6e9ed6e2471 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:08.239667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1029/2036] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:08 2022] POST /identity/v3/projects => generated 379 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:08.245018 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a177c0-5050-4228-a529-4a395848204f None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:08.262976 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a177c0-5050-4228-a529-4a395848204f None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:08.276457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bac75-1e1a-45c7-934b-d6e9ed6e2471 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:08.278301 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-836bac75-1e1a-45c7-934b-d6e9ed6e2471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:08.351744 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbfe45-c259-4ab7-9cc3-64a26faf89a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:08.352581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbfe45-c259-4ab7-9cc3-64a26faf89a8 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:08.352887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbfe45-c259-4ab7-9cc3-64a26faf89a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:08.353274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbfe45-c259-4ab7-9cc3-64a26faf89a8 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:08.354119 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dbfe45-c259-4ab7-9cc3-64a26faf89a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:08.357152 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dbfe45-c259-4ab7-9cc3-64a26faf89a8 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:08.388858 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1008/2037] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:08 2022] POST /identity/v3/users => generated 296 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:08.389510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bac75-1e1a-45c7-934b-d6e9ed6e2471 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:08.398409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de0569-4982-436f-a0ff-d11051462fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:08.399487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbfe45-c259-4ab7-9cc3-64a26faf89a8 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:08.399647 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de0569-4982-436f-a0ff-d11051462fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:08.400317 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de0569-4982-436f-a0ff-d11051462fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:08.400734 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de0569-4982-436f-a0ff-d11051462fb4 admin admin] PATH_INFO: `/v3/users/eb71a8dc3bef478bba83e58384923ab8` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:08.402229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de0569-4982-436f-a0ff-d11051462fb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb71a8dc3bef478bba83e58384923ab8)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:08.404414 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de0569-4982-436f-a0ff-d11051462fb4 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:08.423049 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbfe45-c259-4ab7-9cc3-64a26faf89a8 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:08.439071 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288e564-0459-4bf4-816b-938d173bdd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:08.440127 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288e564-0459-4bf4-816b-938d173bdd85 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:08.442231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288e564-0459-4bf4-816b-938d173bdd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:08.442978 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288e564-0459-4bf4-816b-938d173bdd85 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:08.444286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288e564-0459-4bf4-816b-938d173bdd85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:08.446297 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288e564-0459-4bf4-816b-938d173bdd85 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:08.448079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:08.466210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1009/2038] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:08.467914 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288e564-0459-4bf4-816b-938d173bdd85 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:08.475672 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:08.483155 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288e564-0459-4bf4-816b-938d173bdd85 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:08.490721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedf0cb-e433-445c-a6ee-d668a79e57a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:08.491986 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedf0cb-e433-445c-a6ee-d668a79e57a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:08.493113 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedf0cb-e433-445c-a6ee-d668a79e57a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:08.493452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedf0cb-e433-445c-a6ee-d668a79e57a8 admin admin] PATH_INFO: `/v3/projects/b9a6fd2569d640b0b64a3f9e07ceaebd/users/0178361d94c74741903e243e6fee2a32/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:08.521779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d4de0569-4982-436f-a0ff-d11051462fb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb71a8dc3bef478bba83e58384923ab8'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:08.549941 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedf0cb-e433-445c-a6ee-d668a79e57a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a6fd2569d640b0b64a3f9e07ceaebd, user_id=0178361d94c74741903e243e6fee2a32, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:08.552789 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedf0cb-e433-445c-a6ee-d668a79e57a8 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:08.563854 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d4de0569-4982-436f-a0ff-d11051462fb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb71a8dc3bef478bba83e58384923ab8'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:08.576318 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d4de0569-4982-436f-a0ff-d11051462fb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb71a8dc3bef478bba83e58384923ab8'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:08.585765 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1010/2039] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:08 2022] PUT /identity/v3/projects/b9a6fd2569d640b0b64a3f9e07ceaebd/users/0178361d94c74741903e243e6fee2a32/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:08.588295 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedf0cb-e433-445c-a6ee-d668a79e57a8 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:08.591534 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d4de0569-4982-436f-a0ff-d11051462fb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb71a8dc3bef478bba83e58384923ab8'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:08.592365 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d4de0569-4982-436f-a0ff-d11051462fb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb71a8dc3bef478bba83e58384923ab8'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:08.600788 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1030/2040] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:08 2022] DELETE /identity/v3/users/eb71a8dc3bef478bba83e58384923ab8 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:08.606365 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d69a35-29ff-4c15-9e80-c4aa023720af None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:08.608001 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d69a35-29ff-4c15-9e80-c4aa023720af None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:08.608599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d69a35-29ff-4c15-9e80-c4aa023720af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:08.670897 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedf0cb-e433-445c-a6ee-d668a79e57a8 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:08.674303 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-b2d69a35-29ff-4c15-9e80-c4aa023720af None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:08.682018 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b2d69a35-29ff-4c15-9e80-c4aa023720af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:08.748123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3012701-e601-400f-b936-bf4669c61b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:08.748123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3012701-e601-400f-b936-bf4669c61b6a admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:08.748123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3012701-e601-400f-b936-bf4669c61b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:08.748123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3012701-e601-400f-b936-bf4669c61b6a admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:08.748553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3012701-e601-400f-b936-bf4669c61b6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:08.750170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3012701-e601-400f-b936-bf4669c61b6a admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:08.761950 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1011/2041] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:08.768326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3012701-e601-400f-b936-bf4669c61b6a admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:08.771706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f3012701-e601-400f-b936-bf4669c61b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:08.793434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3012701-e601-400f-b936-bf4669c61b6a admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:08.801411 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a56977-26ed-425e-9e1d-3011b646d774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:08.802881 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a56977-26ed-425e-9e1d-3011b646d774 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:08.803513 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a56977-26ed-425e-9e1d-3011b646d774 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:08.804090 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a56977-26ed-425e-9e1d-3011b646d774 admin admin] PATH_INFO: `/v3/projects/b9a6fd2569d640b0b64a3f9e07ceaebd/users/0178361d94c74741903e243e6fee2a32/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:08.807166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a56977-26ed-425e-9e1d-3011b646d774 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a6fd2569d640b0b64a3f9e07ceaebd, user_id=0178361d94c74741903e243e6fee2a32, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:08.810965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a56977-26ed-425e-9e1d-3011b646d774 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:08.841108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1031/2042] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:08.841326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1012/2043] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:08 2022] PUT /identity/v3/projects/b9a6fd2569d640b0b64a3f9e07ceaebd/users/0178361d94c74741903e243e6fee2a32/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:08.850881 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d69a35-29ff-4c15-9e80-c4aa023720af None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:08.851125 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34200b3-87e8-43c9-b5e3-36186140eb5c None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:08.851442 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34200b3-87e8-43c9-b5e3-36186140eb5c None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:08.851733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34200b3-87e8-43c9-b5e3-36186140eb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:08.853963 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b2d69a35-29ff-4c15-9e80-c4aa023720af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:08.888319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-d34200b3-87e8-43c9-b5e3-36186140eb5c None None] MFA Rules not processed for user `0178361d94c74741903e243e6fee2a32`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:08.890896 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d34200b3-87e8-43c9-b5e3-36186140eb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:08.961389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d69a35-29ff-4c15-9e80-c4aa023720af None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:08.992115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:09.027569 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1013/2044] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:09.041401 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34200b3-87e8-43c9-b5e3-36186140eb5c None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:09.042646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5c23b-0843-49cf-9c34-4a141197eb96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:09.043282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d34200b3-87e8-43c9-b5e3-36186140eb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:09.043486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5c23b-0843-49cf-9c34-4a141197eb96 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:09.043804 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5c23b-0843-49cf-9c34-4a141197eb96 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:09.044088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5c23b-0843-49cf-9c34-4a141197eb96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:09.072058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:09.146666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34200b3-87e8-43c9-b5e3-36186140eb5c None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:09.150691 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a23c1-dd98-4fcc-b086-c25f90827b27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:09.151578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a23c1-dd98-4fcc-b086-c25f90827b27 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:09.151899 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a23c1-dd98-4fcc-b086-c25f90827b27 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:09.152148 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a23c1-dd98-4fcc-b086-c25f90827b27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:09.155467 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5c23b-0843-49cf-9c34-4a141197eb96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:09.157152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d5c23b-0843-49cf-9c34-4a141197eb96 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:09.163267 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a23c1-dd98-4fcc-b086-c25f90827b27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:09.164815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a23c1-dd98-4fcc-b086-c25f90827b27 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:09.168426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1032/2045] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:09.180034 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1014/2046] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:09.231338 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5c23b-0843-49cf-9c34-4a141197eb96 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:09.232872 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-79d5c23b-0843-49cf-9c34-4a141197eb96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:09.246115 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5c23b-0843-49cf-9c34-4a141197eb96 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:09.250726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc463a-f753-4d37-a0a9-a5121fcf492c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:09.251644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc463a-f753-4d37-a0a9-a5121fcf492c service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:09.251996 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc463a-f753-4d37-a0a9-a5121fcf492c service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:09.252362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc463a-f753-4d37-a0a9-a5121fcf492c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:09.254310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e7bc463a-f753-4d37-a0a9-a5121fcf492c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:09.337282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc463a-f753-4d37-a0a9-a5121fcf492c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:09.338853 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc463a-f753-4d37-a0a9-a5121fcf492c service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:09.349069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1033/2047] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:09.491635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a23c1-dd98-4fcc-b086-c25f90827b27 service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:09.493395 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-136a23c1-dd98-4fcc-b086-c25f90827b27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:09.510738 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a23c1-dd98-4fcc-b086-c25f90827b27 service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:09.516413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f1d23-315f-409d-a412-a84006a6e106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:09.517589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f1d23-315f-409d-a412-a84006a6e106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:09.517875 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f1d23-315f-409d-a412-a84006a6e106 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:09.518154 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f1d23-315f-409d-a412-a84006a6e106 admin admin] PATH_INFO: `/v3/projects/91c645733c504bec8ca9b0f82ed5c237` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:09.519700 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f1d23-315f-409d-a412-a84006a6e106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91c645733c504bec8ca9b0f82ed5c237)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:09.523091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f1d23-315f-409d-a412-a84006a6e106 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:09.585266 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cf2f1d23-315f-409d-a412-a84006a6e106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91c645733c504bec8ca9b0f82ed5c237', 'request_id': 'req-cf2f1d23-315f-409d-a412-a84006a6e106'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:09.595096 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cf2f1d23-315f-409d-a412-a84006a6e106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91c645733c504bec8ca9b0f82ed5c237', 'request_id': 'req-cf2f1d23-315f-409d-a412-a84006a6e106'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:09.603276 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cf2f1d23-315f-409d-a412-a84006a6e106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91c645733c504bec8ca9b0f82ed5c237', 'request_id': 'req-cf2f1d23-315f-409d-a412-a84006a6e106'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:09.614539 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cf2f1d23-315f-409d-a412-a84006a6e106 admin admin] The token cache is being invalidate because project 91c645733c504bec8ca9b0f82ed5c237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:09.614838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cf2f1d23-315f-409d-a412-a84006a6e106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:09.615541 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cf2f1d23-315f-409d-a412-a84006a6e106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:09.617775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1015/2048] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:09 2022] DELETE /identity/v3/projects/91c645733c504bec8ca9b0f82ed5c237 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:09.622210 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc463a-f753-4d37-a0a9-a5121fcf492c service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:09.634543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc463a-f753-4d37-a0a9-a5121fcf492c service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:09.700472 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184002c-4855-4a41-9089-5d5478d45b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:09.701581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184002c-4855-4a41-9089-5d5478d45b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:09.701859 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184002c-4855-4a41-9089-5d5478d45b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:09.702143 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184002c-4855-4a41-9089-5d5478d45b60 admin admin] PATH_INFO: `/v3/users/c1c489335ef541408c65e72be71806cb` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:09.703959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184002c-4855-4a41-9089-5d5478d45b60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1c489335ef541408c65e72be71806cb)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:09.705871 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184002c-4855-4a41-9089-5d5478d45b60 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:09.768084 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6184002c-4855-4a41-9089-5d5478d45b60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1c489335ef541408c65e72be71806cb'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:09.770551 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:09.784824 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6184002c-4855-4a41-9089-5d5478d45b60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1c489335ef541408c65e72be71806cb'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:09.794200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6184002c-4855-4a41-9089-5d5478d45b60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1c489335ef541408c65e72be71806cb'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:09.804470 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6184002c-4855-4a41-9089-5d5478d45b60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1c489335ef541408c65e72be71806cb'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:09.805530 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6184002c-4855-4a41-9089-5d5478d45b60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1c489335ef541408c65e72be71806cb'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:09.813336 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1034/2049] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:09 2022] DELETE /identity/v3/users/c1c489335ef541408c65e72be71806cb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:09.926605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f1d23-315f-409d-a412-a84006a6e106 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:09.928209 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-cf2f1d23-315f-409d-a412-a84006a6e106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:09.950214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f1d23-315f-409d-a412-a84006a6e106 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:10.041510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5014342b-52c9-458f-babb-d2568ef240af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:10.045482 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014342b-52c9-458f-babb-d2568ef240af admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:10.045816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014342b-52c9-458f-babb-d2568ef240af admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:10.046098 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014342b-52c9-458f-babb-d2568ef240af admin admin] PATH_INFO: `/v3/users/f69ea1b2d30a4359aa41d87ab8859b00` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:10.047742 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5014342b-52c9-458f-babb-d2568ef240af admin admin] RBAC: Authorizing `identity:delete_user(user_id=f69ea1b2d30a4359aa41d87ab8859b00)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:10.049771 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5014342b-52c9-458f-babb-d2568ef240af admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:10.068771 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:10.114841 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-5014342b-52c9-458f-babb-d2568ef240af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f69ea1b2d30a4359aa41d87ab8859b00'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:10.130325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-5014342b-52c9-458f-babb-d2568ef240af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f69ea1b2d30a4359aa41d87ab8859b00'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:10.141705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-5014342b-52c9-458f-babb-d2568ef240af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f69ea1b2d30a4359aa41d87ab8859b00'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:10.142992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-5014342b-52c9-458f-babb-d2568ef240af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f69ea1b2d30a4359aa41d87ab8859b00'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:10.152959 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-5014342b-52c9-458f-babb-d2568ef240af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f69ea1b2d30a4359aa41d87ab8859b00'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:10.161881 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1016/2050] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:09 2022] DELETE /identity/v3/users/f69ea1b2d30a4359aa41d87ab8859b00 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:10.167419 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b4d45-e771-4913-9eb7-86aaf578aa23 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:10.168246 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b4d45-e771-4913-9eb7-86aaf578aa23 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:10.168656 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b4d45-e771-4913-9eb7-86aaf578aa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:10.208358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-582b4d45-e771-4913-9eb7-86aaf578aa23 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:10.213237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-582b4d45-e771-4913-9eb7-86aaf578aa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:10.336001 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1035/2051] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:10.347588 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5014342b-52c9-458f-babb-d2568ef240af admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:10.349963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-5014342b-52c9-458f-babb-d2568ef240af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:10.431871 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b4d45-e771-4913-9eb7-86aaf578aa23 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:10.434274 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-582b4d45-e771-4913-9eb7-86aaf578aa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:10.449009 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b4d45-e771-4913-9eb7-86aaf578aa23 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:10.454592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cffa2b2-ac24-409a-a18c-bf8f071c34b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:10.455585 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffa2b2-ac24-409a-a18c-bf8f071c34b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:10.455987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffa2b2-ac24-409a-a18c-bf8f071c34b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:10.456267 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffa2b2-ac24-409a-a18c-bf8f071c34b9 admin admin] PATH_INFO: `/v3/projects/9a6d95600cab4b329b6eff2a9e61c6ac` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:10.459403 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cffa2b2-ac24-409a-a18c-bf8f071c34b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a6d95600cab4b329b6eff2a9e61c6ac)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:10.460430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cffa2b2-ac24-409a-a18c-bf8f071c34b9 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:10.477991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5014342b-52c9-458f-babb-d2568ef240af admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:10.483161 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494e118-4281-4191-b925-0238ff2ceb74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:10.483161 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494e118-4281-4191-b925-0238ff2ceb74 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:10.483161 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494e118-4281-4191-b925-0238ff2ceb74 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:10.483594 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494e118-4281-4191-b925-0238ff2ceb74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:10.496780 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494e118-4281-4191-b925-0238ff2ceb74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:10.498576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494e118-4281-4191-b925-0238ff2ceb74 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:10.510867 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1017/2052] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:10.519228 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-9cffa2b2-ac24-409a-a18c-bf8f071c34b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a6d95600cab4b329b6eff2a9e61c6ac', 'request_id': 'req-9cffa2b2-ac24-409a-a18c-bf8f071c34b9'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:10.526430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-9cffa2b2-ac24-409a-a18c-bf8f071c34b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a6d95600cab4b329b6eff2a9e61c6ac', 'request_id': 'req-9cffa2b2-ac24-409a-a18c-bf8f071c34b9'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:10.539771 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-9cffa2b2-ac24-409a-a18c-bf8f071c34b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a6d95600cab4b329b6eff2a9e61c6ac', 'request_id': 'req-9cffa2b2-ac24-409a-a18c-bf8f071c34b9'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:10.558870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-9cffa2b2-ac24-409a-a18c-bf8f071c34b9 admin admin] The token cache is being invalidate because project 9a6d95600cab4b329b6eff2a9e61c6ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:10.559271 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-9cffa2b2-ac24-409a-a18c-bf8f071c34b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:10.560226 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-9cffa2b2-ac24-409a-a18c-bf8f071c34b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:10.562371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1036/2053] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:10 2022] DELETE /identity/v3/projects/9a6d95600cab4b329b6eff2a9e61c6ac => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:10.567175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7544317-e837-4943-aca7-436700c42330 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:10.567777 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7544317-e837-4943-aca7-436700c42330 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:10.568582 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7544317-e837-4943-aca7-436700c42330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:10.596841 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-d7544317-e837-4943-aca7-436700c42330 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:10.598310 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d7544317-e837-4943-aca7-436700c42330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:10.688553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1018/2054] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:10.696495 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d068406-bda9-4cff-a9c0-87aa2c2f47de None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:10.696786 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d068406-bda9-4cff-a9c0-87aa2c2f47de None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:10.697069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d068406-bda9-4cff-a9c0-87aa2c2f47de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:10.717878 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-3d068406-bda9-4cff-a9c0-87aa2c2f47de None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:10.719817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-3d068406-bda9-4cff-a9c0-87aa2c2f47de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:10.731551 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1037/2055] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:10.733719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7544317-e837-4943-aca7-436700c42330 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:10.745160 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7544317-e837-4943-aca7-436700c42330 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:10.750961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ee64a-79b6-4ed5-a9a8-70ef60c54cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:10.751672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ee64a-79b6-4ed5-a9a8-70ef60c54cea admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:10.751956 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ee64a-79b6-4ed5-a9a8-70ef60c54cea admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:10.752227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ee64a-79b6-4ed5-a9a8-70ef60c54cea admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:10.752958 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ee64a-79b6-4ed5-a9a8-70ef60c54cea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:46:10.753266 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ee64a-79b6-4ed5-a9a8-70ef60c54cea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:10.754871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ee64a-79b6-4ed5-a9a8-70ef60c54cea admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:10.772116 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1019/2056] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:46:10.773452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d068406-bda9-4cff-a9c0-87aa2c2f47de None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:10.784165 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d068406-bda9-4cff-a9c0-87aa2c2f47de None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:10.789071 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b4309-f0f1-4f67-b97c-303accc63293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:10.789794 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b4309-f0f1-4f67-b97c-303accc63293 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:10.790379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b4309-f0f1-4f67-b97c-303accc63293 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:10.790495 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b4309-f0f1-4f67-b97c-303accc63293 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:10.791274 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b4309-f0f1-4f67-b97c-303accc63293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:10.793429 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b4309-f0f1-4f67-b97c-303accc63293 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:10.836090 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1038/2057] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:10 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:10.840246 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ee64a-79b6-4ed5-a9a8-70ef60c54cea admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:10.852526 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ee64a-79b6-4ed5-a9a8-70ef60c54cea admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:10.883701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b4309-f0f1-4f67-b97c-303accc63293 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:10.885708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:10.885912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-fd4b4309-f0f1-4f67-b97c-303accc63293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:10.925273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:10.954424 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afd8ca-f97f-426f-909b-aba80accecb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:10.955117 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afd8ca-f97f-426f-909b-aba80accecb7 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:10.955428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afd8ca-f97f-426f-909b-aba80accecb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:10.955898 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8afd8ca-f97f-426f-909b-aba80accecb7 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:10.956396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afd8ca-f97f-426f-909b-aba80accecb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:10.958322 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8afd8ca-f97f-426f-909b-aba80accecb7 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:10.983112 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1020/2058] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:10 2022] POST /identity/v3/users => generated 290 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:46:10.983311 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b4309-f0f1-4f67-b97c-303accc63293 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:10.985449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afd8ca-f97f-426f-909b-aba80accecb7 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:10.989005 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b311c8-8731-46bc-9077-9416633e9efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:10.990896 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b311c8-8731-46bc-9077-9416633e9efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:10.991570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b311c8-8731-46bc-9077-9416633e9efb admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:10.991890 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b311c8-8731-46bc-9077-9416633e9efb admin admin] PATH_INFO: `/v3/projects/154ca230a6744530a3c3eab7653673f1` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:10.993692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b311c8-8731-46bc-9077-9416633e9efb admin admin] RBAC: Authorizing `identity:delete_project(project_id=154ca230a6744530a3c3eab7653673f1)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:10.995433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8afd8ca-f97f-426f-909b-aba80accecb7 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:10.996229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b311c8-8731-46bc-9077-9416633e9efb admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:11.001483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780c078-9d35-47b3-acdd-8d4032f5c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:11.002187 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780c078-9d35-47b3-acdd-8d4032f5c442 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.002458 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780c078-9d35-47b3-acdd-8d4032f5c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.002729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780c078-9d35-47b3-acdd-8d4032f5c442 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.003517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780c078-9d35-47b3-acdd-8d4032f5c442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:11.005110 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780c078-9d35-47b3-acdd-8d4032f5c442 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:11.015686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1021/2059] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:11.019670 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780c078-9d35-47b3-acdd-8d4032f5c442 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:11.029493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780c078-9d35-47b3-acdd-8d4032f5c442 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:11.034232 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f9a85-8947-42b4-b25e-2af4ef35d8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:11.034824 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f9a85-8947-42b4-b25e-2af4ef35d8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.035104 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f9a85-8947-42b4-b25e-2af4ef35d8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.035406 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f9a85-8947-42b4-b25e-2af4ef35d8c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.036128 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f9a85-8947-42b4-b25e-2af4ef35d8c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:11.037691 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f9a85-8947-42b4-b25e-2af4ef35d8c4 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:11.049569 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1022/2060] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:46:11.053511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-74b311c8-8731-46bc-9077-9416633e9efb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '154ca230a6744530a3c3eab7653673f1', 'request_id': 'req-74b311c8-8731-46bc-9077-9416633e9efb'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:11.053663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f9a85-8947-42b4-b25e-2af4ef35d8c4 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:11.059833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-74b311c8-8731-46bc-9077-9416633e9efb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '154ca230a6744530a3c3eab7653673f1', 'request_id': 'req-74b311c8-8731-46bc-9077-9416633e9efb'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:11.063878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f9a85-8947-42b4-b25e-2af4ef35d8c4 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:11.068478 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-74b311c8-8731-46bc-9077-9416633e9efb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '154ca230a6744530a3c3eab7653673f1', 'request_id': 'req-74b311c8-8731-46bc-9077-9416633e9efb'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:11.079706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-74b311c8-8731-46bc-9077-9416633e9efb admin admin] The token cache is being invalidate because project 154ca230a6744530a3c3eab7653673f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:11.080104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-74b311c8-8731-46bc-9077-9416633e9efb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:11.080908 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-74b311c8-8731-46bc-9077-9416633e9efb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:11.082509 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1039/2061] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:10 2022] DELETE /identity/v3/projects/154ca230a6744530a3c3eab7653673f1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:11.087199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657e54b-c004-401b-b0ea-b1cfe96ab5df None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.087509 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657e54b-c004-401b-b0ea-b1cfe96ab5df None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.087792 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657e54b-c004-401b-b0ea-b1cfe96ab5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.108336 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-9657e54b-c004-401b-b0ea-b1cfe96ab5df None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:11.110142 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-9657e54b-c004-401b-b0ea-b1cfe96ab5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:11.122708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ed23d-fb14-43a1-9fcd-8b39128237ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:11.123625 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ed23d-fb14-43a1-9fcd-8b39128237ce admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.123911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ed23d-fb14-43a1-9fcd-8b39128237ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.124186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ed23d-fb14-43a1-9fcd-8b39128237ce admin admin] PATH_INFO: `/v3/projects/317ace7316f64f9fb0ccd69b0928ce54/users/89370292815f4378b712732a031be272/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.139070 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ed23d-fb14-43a1-9fcd-8b39128237ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=317ace7316f64f9fb0ccd69b0928ce54, user_id=89370292815f4378b712732a031be272, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:11.141761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ed23d-fb14-43a1-9fcd-8b39128237ce admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:11.155714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1023/2062] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:11 2022] PUT /identity/v3/projects/317ace7316f64f9fb0ccd69b0928ce54/users/89370292815f4378b712732a031be272/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:11.166633 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63c8e4-7e5c-49bc-adb6-dee36e5b366c None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.166918 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63c8e4-7e5c-49bc-adb6-dee36e5b366c None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.167194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63c8e4-7e5c-49bc-adb6-dee36e5b366c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.172069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1040/2063] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:11.179687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2c4fd-35ff-45ed-acad-e5a454a403c6 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.179981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2c4fd-35ff-45ed-acad-e5a454a403c6 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.180263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd2c4fd-35ff-45ed-acad-e5a454a403c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.194699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-ce63c8e4-7e5c-49bc-adb6-dee36e5b366c None None] MFA Rules not processed for user `89370292815f4378b712732a031be272`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:11.196046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ce63c8e4-7e5c-49bc-adb6-dee36e5b366c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:11.198420 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-3fd2c4fd-35ff-45ed-acad-e5a454a403c6 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:11.199632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-3fd2c4fd-35ff-45ed-acad-e5a454a403c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:11.248161 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:11.266569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1041/2064] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:11.269274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1024/2065] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:11.272682 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2c4fd-35ff-45ed-acad-e5a454a403c6 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:11.273510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63c8e4-7e5c-49bc-adb6-dee36e5b366c None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:11.274853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ce63c8e4-7e5c-49bc-adb6-dee36e5b366c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:11.281522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd2c4fd-35ff-45ed-acad-e5a454a403c6 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:11.283683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:11.286725 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9905789-486c-4c9b-8322-0282f3fdf060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:11.287633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9905789-486c-4c9b-8322-0282f3fdf060 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.288034 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9905789-486c-4c9b-8322-0282f3fdf060 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.288460 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9905789-486c-4c9b-8322-0282f3fdf060 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.289156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9905789-486c-4c9b-8322-0282f3fdf060 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:46:11.289461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9905789-486c-4c9b-8322-0282f3fdf060 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:11.290837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63c8e4-7e5c-49bc-adb6-dee36e5b366c None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:11.291107 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9905789-486c-4c9b-8322-0282f3fdf060 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:11.296727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b71ac5-3de8-4841-ab4b-da5731a860d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:11.297384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b71ac5-3de8-4841-ab4b-da5731a860d5 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.297665 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b71ac5-3de8-4841-ab4b-da5731a860d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.297970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b71ac5-3de8-4841-ab4b-da5731a860d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.298658 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b71ac5-3de8-4841-ab4b-da5731a860d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:11.300673 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b71ac5-3de8-4841-ab4b-da5731a860d5 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:11.303963 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1042/2066] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:46:11.308137 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9905789-486c-4c9b-8322-0282f3fdf060 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:11.320002 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9905789-486c-4c9b-8322-0282f3fdf060 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:11.325752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c1e4e-7aa8-4f3a-b821-2ac7a0a8512a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:11.326582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c1e4e-7aa8-4f3a-b821-2ac7a0a8512a admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.326995 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c1e4e-7aa8-4f3a-b821-2ac7a0a8512a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.327422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c1e4e-7aa8-4f3a-b821-2ac7a0a8512a admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.328217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c1e4e-7aa8-4f3a-b821-2ac7a0a8512a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:11.329273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1025/2067] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:11 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:46:11.330359 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c1e4e-7aa8-4f3a-b821-2ac7a0a8512a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:11.333383 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b71ac5-3de8-4841-ab4b-da5731a860d5 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:11.363545 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1043/2068] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:11 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:46:11.367608 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c1e4e-7aa8-4f3a-b821-2ac7a0a8512a admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:11.413262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b71ac5-3de8-4841-ab4b-da5731a860d5 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:11.448898 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c1e4e-7aa8-4f3a-b821-2ac7a0a8512a admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:11.454571 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731646e5-b8e5-41cb-9915-cf244512c02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:11.455838 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731646e5-b8e5-41cb-9915-cf244512c02e admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.456122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731646e5-b8e5-41cb-9915-cf244512c02e admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.456805 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731646e5-b8e5-41cb-9915-cf244512c02e admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.457767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731646e5-b8e5-41cb-9915-cf244512c02e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:11.460189 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731646e5-b8e5-41cb-9915-cf244512c02e admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:11.491137 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1044/2069] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:11 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:46:11.496277 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731646e5-b8e5-41cb-9915-cf244512c02e admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:11.504898 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7a843-7824-45f4-99c9-3829a5f76a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:11.505767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7a843-7824-45f4-99c9-3829a5f76a90 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.505889 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7a843-7824-45f4-99c9-3829a5f76a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.506213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7a843-7824-45f4-99c9-3829a5f76a90 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.506915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7a843-7824-45f4-99c9-3829a5f76a90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:11.509411 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7a843-7824-45f4-99c9-3829a5f76a90 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:11.513619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731646e5-b8e5-41cb-9915-cf244512c02e admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:11.522175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc5173-9c85-403e-b31f-cc10d8c711f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:11.523093 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc5173-9c85-403e-b31f-cc10d8c711f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.523698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc5173-9c85-403e-b31f-cc10d8c711f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.524435 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc5173-9c85-403e-b31f-cc10d8c711f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.525913 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc5173-9c85-403e-b31f-cc10d8c711f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:11.528786 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc5173-9c85-403e-b31f-cc10d8c711f4 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:11.541360 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1026/2070] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:11 2022] POST /identity/v3/users => generated 296 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:46:11.542571 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7a843-7824-45f4-99c9-3829a5f76a90 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:11.545521 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1045/2071] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:11.548358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc5173-9c85-403e-b31f-cc10d8c711f4 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:11.557312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7a843-7824-45f4-99c9-3829a5f76a90 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:11.558485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc5173-9c85-403e-b31f-cc10d8c711f4 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:11.562343 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce5358-a818-40aa-a3ab-5edc8fd9334b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:11.563017 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce5358-a818-40aa-a3ab-5edc8fd9334b admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.563305 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce5358-a818-40aa-a3ab-5edc8fd9334b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.563615 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce5358-a818-40aa-a3ab-5edc8fd9334b admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.564352 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce5358-a818-40aa-a3ab-5edc8fd9334b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:11.564764 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3c21a-08d3-4f49-bee9-9c41550ef055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:11.565417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3c21a-08d3-4f49-bee9-9c41550ef055 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.565730 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3c21a-08d3-4f49-bee9-9c41550ef055 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.565960 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce5358-a818-40aa-a3ab-5edc8fd9334b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:11.566061 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3c21a-08d3-4f49-bee9-9c41550ef055 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.566724 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3c21a-08d3-4f49-bee9-9c41550ef055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:11.569108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3c21a-08d3-4f49-bee9-9c41550ef055 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:11.578384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1027/2072] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:11.585321 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce5358-a818-40aa-a3ab-5edc8fd9334b admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:11.587187 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1046/2073] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:11.588266 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3c21a-08d3-4f49-bee9-9c41550ef055 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:11.597483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce5358-a818-40aa-a3ab-5edc8fd9334b admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:11.600478 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3c21a-08d3-4f49-bee9-9c41550ef055 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:11.602829 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebd367-9905-47b6-8cc3-e9861b223f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:11.603834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebd367-9905-47b6-8cc3-e9861b223f0f admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.604526 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebd367-9905-47b6-8cc3-e9861b223f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.604898 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebd367-9905-47b6-8cc3-e9861b223f0f admin admin] PATH_INFO: `/v3/projects/191cd32ce6fb4a3d989e400dc1d55301/users/a2b12d76a2cc435db73d8f691e8212bb/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.605656 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45360dc4-5263-485d-84b1-b04c0c9a61ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:11.616031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45360dc4-5263-485d-84b1-b04c0c9a61ba admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.616661 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45360dc4-5263-485d-84b1-b04c0c9a61ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.619893 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45360dc4-5263-485d-84b1-b04c0c9a61ba admin admin] PATH_INFO: `/v3/projects/0a91f4b2bf5e49adb47b96bf2d99af1b/users/dbbcf057a4744909bea506f6a7da003b/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.647645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45360dc4-5263-485d-84b1-b04c0c9a61ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a91f4b2bf5e49adb47b96bf2d99af1b, user_id=dbbcf057a4744909bea506f6a7da003b, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:11.651784 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45360dc4-5263-485d-84b1-b04c0c9a61ba admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:11.652712 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebd367-9905-47b6-8cc3-e9861b223f0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=191cd32ce6fb4a3d989e400dc1d55301, user_id=a2b12d76a2cc435db73d8f691e8212bb, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:11.655564 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baebd367-9905-47b6-8cc3-e9861b223f0f admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:11.674114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1047/2074] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:11 2022] PUT /identity/v3/projects/0a91f4b2bf5e49adb47b96bf2d99af1b/users/dbbcf057a4744909bea506f6a7da003b/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:11.686516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5eee9-5d7c-4b19-a525-ca8a15aebfe7 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.687040 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5eee9-5d7c-4b19-a525-ca8a15aebfe7 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.687040 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5eee9-5d7c-4b19-a525-ca8a15aebfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.693019 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1028/2075] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:11 2022] PUT /identity/v3/projects/191cd32ce6fb4a3d989e400dc1d55301/users/a2b12d76a2cc435db73d8f691e8212bb/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:11.697913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebd367-9905-47b6-8cc3-e9861b223f0f admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:11.724475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d3e5eee9-5d7c-4b19-a525-ca8a15aebfe7 None None] MFA Rules not processed for user `dbbcf057a4744909bea506f6a7da003b`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:11.725983 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d3e5eee9-5d7c-4b19-a525-ca8a15aebfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:11.782487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:11.807810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebd367-9905-47b6-8cc3-e9861b223f0f admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:11.813147 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c286d-cf29-46d4-b33c-23bc2cd71309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:11.813765 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c286d-cf29-46d4-b33c-23bc2cd71309 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.814057 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c286d-cf29-46d4-b33c-23bc2cd71309 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.814335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c286d-cf29-46d4-b33c-23bc2cd71309 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.815074 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c286d-cf29-46d4-b33c-23bc2cd71309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:11.816705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c286d-cf29-46d4-b33c-23bc2cd71309 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:11.831839 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:11.832526 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1029/2076] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:11.833358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c286d-cf29-46d4-b33c-23bc2cd71309 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:11.845693 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c286d-cf29-46d4-b33c-23bc2cd71309 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:11.850846 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ed1a9-1af9-48ca-a168-751bf01877ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:11.851793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ed1a9-1af9-48ca-a168-751bf01877ea admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.852080 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ed1a9-1af9-48ca-a168-751bf01877ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.852516 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ed1a9-1af9-48ca-a168-751bf01877ea admin admin] PATH_INFO: `/v3/projects/191cd32ce6fb4a3d989e400dc1d55301/users/a2b12d76a2cc435db73d8f691e8212bb/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.854608 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32ed1a9-1af9-48ca-a168-751bf01877ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=191cd32ce6fb4a3d989e400dc1d55301, user_id=a2b12d76a2cc435db73d8f691e8212bb, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:11.857391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32ed1a9-1af9-48ca-a168-751bf01877ea admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:11.858979 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1048/2077] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:11.868678 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5eee9-5d7c-4b19-a525-ca8a15aebfe7 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:11.895887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1030/2078] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:11 2022] PUT /identity/v3/projects/191cd32ce6fb4a3d989e400dc1d55301/users/a2b12d76a2cc435db73d8f691e8212bb/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:11.902621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5eee9-5d7c-4b19-a525-ca8a15aebfe7 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:11.905548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09286b4-f4c0-4270-aa9c-d7dcb959ac83 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:11.906727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09286b4-f4c0-4270-aa9c-d7dcb959ac83 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:11.907131 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09286b4-f4c0-4270-aa9c-d7dcb959ac83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:11.959093 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-d09286b4-f4c0-4270-aa9c-d7dcb959ac83 None None] MFA Rules not processed for user `a2b12d76a2cc435db73d8f691e8212bb`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:11.960865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d09286b4-f4c0-4270-aa9c-d7dcb959ac83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:12.014284 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad2a43-75a6-4c4b-a519-96a3912d07a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:12.015812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad2a43-75a6-4c4b-a519-96a3912d07a3 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:12.016103 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad2a43-75a6-4c4b-a519-96a3912d07a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:12.016471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bad2a43-75a6-4c4b-a519-96a3912d07a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:12.017712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad2a43-75a6-4c4b-a519-96a3912d07a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:12.020284 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bad2a43-75a6-4c4b-a519-96a3912d07a3 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:12.040433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1031/2079] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:12.057961 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1049/2080] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:11 2022] POST /identity/v3/projects => generated 375 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:12.062537 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09286b4-f4c0-4270-aa9c-d7dcb959ac83 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:12.076667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad2a43-75a6-4c4b-a519-96a3912d07a3 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:12.078560 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09286b4-f4c0-4270-aa9c-d7dcb959ac83 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:12.079163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-6bad2a43-75a6-4c4b-a519-96a3912d07a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:12.139947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bad2a43-75a6-4c4b-a519-96a3912d07a3 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:12.144691 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8954deb-a563-4de3-bba8-4000bcab8f5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:12.145397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8954deb-a563-4de3-bba8-4000bcab8f5d service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:12.145639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8954deb-a563-4de3-bba8-4000bcab8f5d service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:12.145918 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8954deb-a563-4de3-bba8-4000bcab8f5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:12.152287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89cabcc-bc2a-4a3e-915b-b39a06abf30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:12.153196 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cabcc-bc2a-4a3e-915b-b39a06abf30e admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:12.153559 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cabcc-bc2a-4a3e-915b-b39a06abf30e admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:12.153943 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cabcc-bc2a-4a3e-915b-b39a06abf30e admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:12.154739 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89cabcc-bc2a-4a3e-915b-b39a06abf30e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:12.156777 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89cabcc-bc2a-4a3e-915b-b39a06abf30e admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:12.181605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1032/2081] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:12 2022] POST /identity/v3/users => generated 294 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:12.185247 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89cabcc-bc2a-4a3e-915b-b39a06abf30e admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:12.196192 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89cabcc-bc2a-4a3e-915b-b39a06abf30e admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:12.201385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824db471-10e0-40e7-bf2b-791cff214a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:12.202047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824db471-10e0-40e7-bf2b-791cff214a38 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:12.202931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824db471-10e0-40e7-bf2b-791cff214a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:12.203205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824db471-10e0-40e7-bf2b-791cff214a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:12.203968 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824db471-10e0-40e7-bf2b-791cff214a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:12.205558 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824db471-10e0-40e7-bf2b-791cff214a38 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:12.219142 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1033/2082] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:46:12.223723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824db471-10e0-40e7-bf2b-791cff214a38 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:12.230813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8954deb-a563-4de3-bba8-4000bcab8f5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:12.233954 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8954deb-a563-4de3-bba8-4000bcab8f5d service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:12.242238 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824db471-10e0-40e7-bf2b-791cff214a38 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:12.248384 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1050/2083] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:12.250493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bb098-eecd-4249-ab29-209462497258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:12.251891 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bb098-eecd-4249-ab29-209462497258 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:12.252117 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bb098-eecd-4249-ab29-209462497258 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:12.252419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bb098-eecd-4249-ab29-209462497258 admin admin] PATH_INFO: `/v3/projects/4e4d9018302f48f7bd90d21deb3b9895/users/2673b3a107854e63959bb4e8dbaae2f6/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:12.282605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bb098-eecd-4249-ab29-209462497258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e4d9018302f48f7bd90d21deb3b9895, user_id=2673b3a107854e63959bb4e8dbaae2f6, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:12.289886 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bb098-eecd-4249-ab29-209462497258 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:12.321043 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1034/2084] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:12 2022] PUT /identity/v3/projects/4e4d9018302f48f7bd90d21deb3b9895/users/2673b3a107854e63959bb4e8dbaae2f6/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:12.332300 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8954deb-a563-4de3-bba8-4000bcab8f5d service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:12.348235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8954deb-a563-4de3-bba8-4000bcab8f5d service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:12.397921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:12.472393 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa184c5-a843-4795-8928-2631360fa7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:12.473034 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa184c5-a843-4795-8928-2631360fa7e2 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:12.473350 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa184c5-a843-4795-8928-2631360fa7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:12.473593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa184c5-a843-4795-8928-2631360fa7e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:12.474335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa184c5-a843-4795-8928-2631360fa7e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:12.476034 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa184c5-a843-4795-8928-2631360fa7e2 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:12.490002 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1051/2085] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:12.491958 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bb098-eecd-4249-ab29-209462497258 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:12.502059 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:12.507816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bb098-eecd-4249-ab29-209462497258 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:12.514812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479005a-a02c-4d90-812a-58eb31880997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:12.515755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479005a-a02c-4d90-812a-58eb31880997 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:12.516400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479005a-a02c-4d90-812a-58eb31880997 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:12.516736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479005a-a02c-4d90-812a-58eb31880997 admin admin] PATH_INFO: `/v3/projects/4e4d9018302f48f7bd90d21deb3b9895/users/2673b3a107854e63959bb4e8dbaae2f6/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:12.520032 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479005a-a02c-4d90-812a-58eb31880997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e4d9018302f48f7bd90d21deb3b9895, user_id=2673b3a107854e63959bb4e8dbaae2f6, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:12.524978 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479005a-a02c-4d90-812a-58eb31880997 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:12.557740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1035/2086] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:12 2022] PUT /identity/v3/projects/4e4d9018302f48f7bd90d21deb3b9895/users/2673b3a107854e63959bb4e8dbaae2f6/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:12.573031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593c229-01c1-42fd-8910-07c137b6bd67 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:12.573326 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593c229-01c1-42fd-8910-07c137b6bd67 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:12.573607 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593c229-01c1-42fd-8910-07c137b6bd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:12.607495 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d593c229-01c1-42fd-8910-07c137b6bd67 None None] MFA Rules not processed for user `2673b3a107854e63959bb4e8dbaae2f6`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:12.613842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d593c229-01c1-42fd-8910-07c137b6bd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:12.722034 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1052/2087] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:12 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:12.748412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479005a-a02c-4d90-812a-58eb31880997 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:12.833310 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479005a-a02c-4d90-812a-58eb31880997 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:12.837924 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720fc97-ee85-4829-bb3a-af615793196c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:12.838736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720fc97-ee85-4829-bb3a-af615793196c service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:12.839067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720fc97-ee85-4829-bb3a-af615793196c service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:12.839369 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720fc97-ee85-4829-bb3a-af615793196c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:12.932371 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720fc97-ee85-4829-bb3a-af615793196c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:12.934251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720fc97-ee85-4829-bb3a-af615793196c service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:12.952556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1036/2088] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:13.027072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593c229-01c1-42fd-8910-07c137b6bd67 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:13.117988 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593c229-01c1-42fd-8910-07c137b6bd67 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:13.123032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6541c-6271-48ab-b9d1-71a525dccb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:13.123999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6541c-6271-48ab-b9d1-71a525dccb95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:13.124571 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6541c-6271-48ab-b9d1-71a525dccb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:13.124931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6541c-6271-48ab-b9d1-71a525dccb95 admin admin] PATH_INFO: `/v3/users/89370292815f4378b712732a031be272` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:13.126235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6541c-6271-48ab-b9d1-71a525dccb95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89370292815f4378b712732a031be272)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:13.128288 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6541c-6271-48ab-b9d1-71a525dccb95 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:13.186821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-35a6541c-6271-48ab-b9d1-71a525dccb95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89370292815f4378b712732a031be272'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:13.189012 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:13.204276 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-35a6541c-6271-48ab-b9d1-71a525dccb95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89370292815f4378b712732a031be272'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:13.213480 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-35a6541c-6271-48ab-b9d1-71a525dccb95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89370292815f4378b712732a031be272'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:13.222840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-35a6541c-6271-48ab-b9d1-71a525dccb95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89370292815f4378b712732a031be272'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:13.223754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-35a6541c-6271-48ab-b9d1-71a525dccb95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89370292815f4378b712732a031be272'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:13.230328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1053/2089] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:13 2022] DELETE /identity/v3/users/89370292815f4378b712732a031be272 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:13.235626 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da21bf-ee8e-48f7-83c7-604353b78f7f None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:13.236161 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da21bf-ee8e-48f7-83c7-604353b78f7f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:13.236294 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da21bf-ee8e-48f7-83c7-604353b78f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:13.304554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c3da21bf-ee8e-48f7-83c7-604353b78f7f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:13.305872 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c3da21bf-ee8e-48f7-83c7-604353b78f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:13.393804 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1037/2090] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:13.408005 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6541c-6271-48ab-b9d1-71a525dccb95 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:13.411680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-35a6541c-6271-48ab-b9d1-71a525dccb95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:13.484149 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6541c-6271-48ab-b9d1-71a525dccb95 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:13.488805 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fef772-64ca-42fe-a36d-51b482573bd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:13.489984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fef772-64ca-42fe-a36d-51b482573bd8 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:13.490427 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fef772-64ca-42fe-a36d-51b482573bd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:13.490948 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fef772-64ca-42fe-a36d-51b482573bd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:13.502721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fef772-64ca-42fe-a36d-51b482573bd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:13.504224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fef772-64ca-42fe-a36d-51b482573bd8 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:13.516916 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1054/2091] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:13.805517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da21bf-ee8e-48f7-83c7-604353b78f7f None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:13.807162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c3da21bf-ee8e-48f7-83c7-604353b78f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:13.821482 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da21bf-ee8e-48f7-83c7-604353b78f7f None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:13.827122 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14a774-49ac-4747-bcdd-f4d7f8e6c768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:13.829034 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14a774-49ac-4747-bcdd-f4d7f8e6c768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:13.829034 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14a774-49ac-4747-bcdd-f4d7f8e6c768 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:13.829273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14a774-49ac-4747-bcdd-f4d7f8e6c768 admin admin] PATH_INFO: `/v3/projects/317ace7316f64f9fb0ccd69b0928ce54` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:13.830377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f14a774-49ac-4747-bcdd-f4d7f8e6c768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=317ace7316f64f9fb0ccd69b0928ce54)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:13.832344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f14a774-49ac-4747-bcdd-f4d7f8e6c768 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:13.874870 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:13.884947 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-7f14a774-49ac-4747-bcdd-f4d7f8e6c768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '317ace7316f64f9fb0ccd69b0928ce54', 'request_id': 'req-7f14a774-49ac-4747-bcdd-f4d7f8e6c768'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:13.896261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-7f14a774-49ac-4747-bcdd-f4d7f8e6c768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '317ace7316f64f9fb0ccd69b0928ce54', 'request_id': 'req-7f14a774-49ac-4747-bcdd-f4d7f8e6c768'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:13.903030 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-7f14a774-49ac-4747-bcdd-f4d7f8e6c768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '317ace7316f64f9fb0ccd69b0928ce54', 'request_id': 'req-7f14a774-49ac-4747-bcdd-f4d7f8e6c768'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:13.914954 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-7f14a774-49ac-4747-bcdd-f4d7f8e6c768 admin admin] The token cache is being invalidate because project 317ace7316f64f9fb0ccd69b0928ce54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:13.915340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-7f14a774-49ac-4747-bcdd-f4d7f8e6c768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:13.916139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-7f14a774-49ac-4747-bcdd-f4d7f8e6c768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:13.917891 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1038/2092] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:13 2022] DELETE /identity/v3/projects/317ace7316f64f9fb0ccd69b0928ce54 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:13.922509 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fef772-64ca-42fe-a36d-51b482573bd8 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:13.932538 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fef772-64ca-42fe-a36d-51b482573bd8 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:13.992047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b108e9-f215-43eb-823b-930c5248a2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:13.993018 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b108e9-f215-43eb-823b-930c5248a2f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:13.993303 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b108e9-f215-43eb-823b-930c5248a2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:13.993576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b108e9-f215-43eb-823b-930c5248a2f6 admin admin] PATH_INFO: `/v3/users/a2b12d76a2cc435db73d8f691e8212bb` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:13.994943 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b108e9-f215-43eb-823b-930c5248a2f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2b12d76a2cc435db73d8f691e8212bb)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:13.997000 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b108e9-f215-43eb-823b-930c5248a2f6 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:14.054241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b6b108e9-f215-43eb-823b-930c5248a2f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2b12d76a2cc435db73d8f691e8212bb'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:14.066840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b6b108e9-f215-43eb-823b-930c5248a2f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2b12d76a2cc435db73d8f691e8212bb'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:14.074368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b6b108e9-f215-43eb-823b-930c5248a2f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2b12d76a2cc435db73d8f691e8212bb'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:14.075682 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:14.084135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b6b108e9-f215-43eb-823b-930c5248a2f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2b12d76a2cc435db73d8f691e8212bb'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:14.085176 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b6b108e9-f215-43eb-823b-930c5248a2f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2b12d76a2cc435db73d8f691e8212bb'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:14.092157 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1055/2093] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:13 2022] DELETE /identity/v3/users/a2b12d76a2cc435db73d8f691e8212bb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:14.387412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14a774-49ac-4747-bcdd-f4d7f8e6c768 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:14.388969 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-7f14a774-49ac-4747-bcdd-f4d7f8e6c768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:14.402511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f14a774-49ac-4747-bcdd-f4d7f8e6c768 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:14.466241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c91548-9966-4b5d-8b1d-0b0a3b8c2d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:14.467094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c91548-9966-4b5d-8b1d-0b0a3b8c2d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:14.467398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c91548-9966-4b5d-8b1d-0b0a3b8c2d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:14.467679 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c91548-9966-4b5d-8b1d-0b0a3b8c2d53 admin admin] PATH_INFO: `/v3/projects/191cd32ce6fb4a3d989e400dc1d55301` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:14.469365 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c91548-9966-4b5d-8b1d-0b0a3b8c2d53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=191cd32ce6fb4a3d989e400dc1d55301)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:14.471257 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c91548-9966-4b5d-8b1d-0b0a3b8c2d53 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:14.523942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-f6c91548-9966-4b5d-8b1d-0b0a3b8c2d53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '191cd32ce6fb4a3d989e400dc1d55301', 'request_id': 'req-f6c91548-9966-4b5d-8b1d-0b0a3b8c2d53'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:14.532356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-f6c91548-9966-4b5d-8b1d-0b0a3b8c2d53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '191cd32ce6fb4a3d989e400dc1d55301', 'request_id': 'req-f6c91548-9966-4b5d-8b1d-0b0a3b8c2d53'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:14.539132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-f6c91548-9966-4b5d-8b1d-0b0a3b8c2d53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '191cd32ce6fb4a3d989e400dc1d55301', 'request_id': 'req-f6c91548-9966-4b5d-8b1d-0b0a3b8c2d53'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:14.550818 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-f6c91548-9966-4b5d-8b1d-0b0a3b8c2d53 admin admin] The token cache is being invalidate because project 191cd32ce6fb4a3d989e400dc1d55301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:14.551108 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-f6c91548-9966-4b5d-8b1d-0b0a3b8c2d53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:14.551794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-f6c91548-9966-4b5d-8b1d-0b0a3b8c2d53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:14.553356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1039/2094] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:14 2022] DELETE /identity/v3/projects/191cd32ce6fb4a3d989e400dc1d55301 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:14.558192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57f67d-0d40-4044-9ad0-4825111e2b4c None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:14.558676 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57f67d-0d40-4044-9ad0-4825111e2b4c None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:14.558928 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57f67d-0d40-4044-9ad0-4825111e2b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:14.577062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-cf57f67d-0d40-4044-9ad0-4825111e2b4c None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:14.578437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-cf57f67d-0d40-4044-9ad0-4825111e2b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:14.641097 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1056/2095] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:14.649145 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801f81b-ac63-4c43-ba77-6737bd319480 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:14.649440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801f81b-ac63-4c43-ba77-6737bd319480 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:14.649718 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801f81b-ac63-4c43-ba77-6737bd319480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:14.669200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-9801f81b-ac63-4c43-ba77-6737bd319480 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:14.670395 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-9801f81b-ac63-4c43-ba77-6737bd319480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:14.678347 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1040/2096] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:14.682587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57f67d-0d40-4044-9ad0-4825111e2b4c None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:14.694718 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57f67d-0d40-4044-9ad0-4825111e2b4c None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:14.700511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7fafb-ec94-440b-85fc-33807dad251a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:14.701220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7fafb-ec94-440b-85fc-33807dad251a admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:14.701502 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7fafb-ec94-440b-85fc-33807dad251a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:14.701779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7fafb-ec94-440b-85fc-33807dad251a admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:14.703472 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7fafb-ec94-440b-85fc-33807dad251a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:46:14.703783 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7fafb-ec94-440b-85fc-33807dad251a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:14.705378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7fafb-ec94-440b-85fc-33807dad251a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:14.717064 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1057/2097] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:46:14.721992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801f81b-ac63-4c43-ba77-6737bd319480 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:14.730984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801f81b-ac63-4c43-ba77-6737bd319480 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:14.735945 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799fc43f-5fd8-4637-9543-b659a26974b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:14.736587 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799fc43f-5fd8-4637-9543-b659a26974b8 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:14.736866 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799fc43f-5fd8-4637-9543-b659a26974b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:14.737140 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799fc43f-5fd8-4637-9543-b659a26974b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:14.737863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799fc43f-5fd8-4637-9543-b659a26974b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:14.739785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799fc43f-5fd8-4637-9543-b659a26974b8 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:14.765732 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1041/2098] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:14 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:14.770136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7fafb-ec94-440b-85fc-33807dad251a admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:14.780255 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7fafb-ec94-440b-85fc-33807dad251a admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:14.835381 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1965dc-c149-4672-af4f-b8ca28bb095d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:14.836043 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1965dc-c149-4672-af4f-b8ca28bb095d admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:14.836329 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1965dc-c149-4672-af4f-b8ca28bb095d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:14.836605 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1965dc-c149-4672-af4f-b8ca28bb095d admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:14.837300 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1965dc-c149-4672-af4f-b8ca28bb095d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:14.839225 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1965dc-c149-4672-af4f-b8ca28bb095d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:14.859148 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1058/2099] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:14 2022] POST /identity/v3/users => generated 292 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:14.863268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799fc43f-5fd8-4637-9543-b659a26974b8 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:14.873781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799fc43f-5fd8-4637-9543-b659a26974b8 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:14.878639 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101aa62-832d-4836-9a37-68493c99dc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:14.879376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101aa62-832d-4836-9a37-68493c99dc4b admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:14.879653 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101aa62-832d-4836-9a37-68493c99dc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:14.879930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101aa62-832d-4836-9a37-68493c99dc4b admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:14.880677 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101aa62-832d-4836-9a37-68493c99dc4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:14.882286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101aa62-832d-4836-9a37-68493c99dc4b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:14.894677 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1042/2100] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:14.898788 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1965dc-c149-4672-af4f-b8ca28bb095d admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:14.909411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1965dc-c149-4672-af4f-b8ca28bb095d admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:14.914206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9d212-74cd-41a4-9fbf-00428ffe7d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:14.914838 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9d212-74cd-41a4-9fbf-00428ffe7d34 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:14.915123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9d212-74cd-41a4-9fbf-00428ffe7d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:14.915436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9d212-74cd-41a4-9fbf-00428ffe7d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:14.916234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9d212-74cd-41a4-9fbf-00428ffe7d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:14.917841 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9d212-74cd-41a4-9fbf-00428ffe7d34 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:14.930498 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1059/2101] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:14.934747 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101aa62-832d-4836-9a37-68493c99dc4b admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:14.945127 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101aa62-832d-4836-9a37-68493c99dc4b admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:14.949865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded19ea7-6348-4031-a01c-87007140e958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:14.950790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded19ea7-6348-4031-a01c-87007140e958 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:14.951078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded19ea7-6348-4031-a01c-87007140e958 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:14.951374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded19ea7-6348-4031-a01c-87007140e958 admin admin] PATH_INFO: `/v3/projects/cb56dbfeb9434e039c9708b86882f4ec/users/9d221d89ac944f21a84331c084b05ced/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:14.966422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded19ea7-6348-4031-a01c-87007140e958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb56dbfeb9434e039c9708b86882f4ec, user_id=9d221d89ac944f21a84331c084b05ced, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:14.969728 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded19ea7-6348-4031-a01c-87007140e958 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:14.972850 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:14.986822 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1043/2102] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:14 2022] PUT /identity/v3/projects/cb56dbfeb9434e039c9708b86882f4ec/users/9d221d89ac944f21a84331c084b05ced/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:14.998533 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6400f4a-217f-4499-912a-f7b782507f77 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:14.998820 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6400f4a-217f-4499-912a-f7b782507f77 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:14.999099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6400f4a-217f-4499-912a-f7b782507f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:15.027966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-c6400f4a-217f-4499-912a-f7b782507f77 None None] MFA Rules not processed for user `9d221d89ac944f21a84331c084b05ced`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:15.029432 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-c6400f4a-217f-4499-912a-f7b782507f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:15.073399 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:15.095944 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1060/2103] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:15.100603 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded19ea7-6348-4031-a01c-87007140e958 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:15.167231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded19ea7-6348-4031-a01c-87007140e958 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:15.171833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56284da-436c-4a9e-848c-1f75e422218f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:15.172502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56284da-436c-4a9e-848c-1f75e422218f admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:15.172787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56284da-436c-4a9e-848c-1f75e422218f admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:15.173066 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56284da-436c-4a9e-848c-1f75e422218f admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:15.173851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56284da-436c-4a9e-848c-1f75e422218f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:15.175786 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56284da-436c-4a9e-848c-1f75e422218f admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:15.203882 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1044/2104] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:15 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:15.207843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6400f4a-217f-4499-912a-f7b782507f77 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:15.219666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6400f4a-217f-4499-912a-f7b782507f77 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:15.277145 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea357b-dc6c-435e-9b98-020fdb3691ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:15.277895 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea357b-dc6c-435e-9b98-020fdb3691ee admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:15.278370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea357b-dc6c-435e-9b98-020fdb3691ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:15.278712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea357b-dc6c-435e-9b98-020fdb3691ee admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:15.279514 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea357b-dc6c-435e-9b98-020fdb3691ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:15.281529 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea357b-dc6c-435e-9b98-020fdb3691ee admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:15.303397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1061/2105] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:15 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:46:15.305793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56284da-436c-4a9e-848c-1f75e422218f admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:15.317042 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56284da-436c-4a9e-848c-1f75e422218f admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:15.321694 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117dab5-19ba-451f-be55-3245118ac98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:15.322359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117dab5-19ba-451f-be55-3245118ac98f admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:15.322641 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117dab5-19ba-451f-be55-3245118ac98f admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:15.322918 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117dab5-19ba-451f-be55-3245118ac98f admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:15.323681 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117dab5-19ba-451f-be55-3245118ac98f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:15.325274 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117dab5-19ba-451f-be55-3245118ac98f admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:15.336963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1045/2106] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:15.341158 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea357b-dc6c-435e-9b98-020fdb3691ee admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:15.352361 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea357b-dc6c-435e-9b98-020fdb3691ee admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:15.365387 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d25a2d-d2a6-48a8-95a5-41e42c89733b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:15.366282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d25a2d-d2a6-48a8-95a5-41e42c89733b admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:15.366565 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d25a2d-d2a6-48a8-95a5-41e42c89733b admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:15.366845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d25a2d-d2a6-48a8-95a5-41e42c89733b admin admin] PATH_INFO: `/v3/projects/18f2f01634f34582992937d1beeaadf7/users/8e78427b641a415482dfbaabcd7812ab/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:15.382767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d25a2d-d2a6-48a8-95a5-41e42c89733b admin admin] RBAC: Authorizing `identity:create_grant(project_id=18f2f01634f34582992937d1beeaadf7, user_id=8e78427b641a415482dfbaabcd7812ab, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:15.385709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d25a2d-d2a6-48a8-95a5-41e42c89733b admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:15.400530 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1062/2107] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:15 2022] PUT /identity/v3/projects/18f2f01634f34582992937d1beeaadf7/users/8e78427b641a415482dfbaabcd7812ab/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:15.404424 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117dab5-19ba-451f-be55-3245118ac98f admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:15.414706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117dab5-19ba-451f-be55-3245118ac98f admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:15.474304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a49d95-9472-49c2-adef-d641901a7635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:15.474976 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a49d95-9472-49c2-adef-d641901a7635 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:15.475260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a49d95-9472-49c2-adef-d641901a7635 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:15.475567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a49d95-9472-49c2-adef-d641901a7635 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:15.476302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a49d95-9472-49c2-adef-d641901a7635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:15.477892 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a49d95-9472-49c2-adef-d641901a7635 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:15.488311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1046/2108] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:15.492461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d25a2d-d2a6-48a8-95a5-41e42c89733b admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:15.503259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d25a2d-d2a6-48a8-95a5-41e42c89733b admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:15.507870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9453b6e-2b71-4fb2-941e-72eaaca1a136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:15.508774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9453b6e-2b71-4fb2-941e-72eaaca1a136 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:15.509057 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9453b6e-2b71-4fb2-941e-72eaaca1a136 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:15.509330 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9453b6e-2b71-4fb2-941e-72eaaca1a136 admin admin] PATH_INFO: `/v3/projects/18f2f01634f34582992937d1beeaadf7/users/8e78427b641a415482dfbaabcd7812ab/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:15.511329 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9453b6e-2b71-4fb2-941e-72eaaca1a136 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18f2f01634f34582992937d1beeaadf7, user_id=8e78427b641a415482dfbaabcd7812ab, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:15.514009 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9453b6e-2b71-4fb2-941e-72eaaca1a136 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:15.524728 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1063/2109] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:15 2022] PUT /identity/v3/projects/18f2f01634f34582992937d1beeaadf7/users/8e78427b641a415482dfbaabcd7812ab/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:15.535047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9e28f-ced1-48b5-a9ec-7ff6a2b3a9ee None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:15.535338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9e28f-ced1-48b5-a9ec-7ff6a2b3a9ee None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:15.535659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9e28f-ced1-48b5-a9ec-7ff6a2b3a9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:15.562272 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-e3f9e28f-ced1-48b5-a9ec-7ff6a2b3a9ee None None] MFA Rules not processed for user `8e78427b641a415482dfbaabcd7812ab`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:15.563524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e3f9e28f-ced1-48b5-a9ec-7ff6a2b3a9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:15.612085 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:15.631381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1047/2110] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:15.641893 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9453b6e-2b71-4fb2-941e-72eaaca1a136 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:15.644163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-c9453b6e-2b71-4fb2-941e-72eaaca1a136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:15.704284 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9453b6e-2b71-4fb2-941e-72eaaca1a136 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:15.709020 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c4e79-6561-4067-8fb1-f646d44a21c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:15.709616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c4e79-6561-4067-8fb1-f646d44a21c8 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:15.709900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c4e79-6561-4067-8fb1-f646d44a21c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:15.710177 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c4e79-6561-4067-8fb1-f646d44a21c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:15.769136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:15.782071 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c4e79-6561-4067-8fb1-f646d44a21c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:15.783551 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c4e79-6561-4067-8fb1-f646d44a21c8 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:15.793058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1064/2111] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:18.949077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9e28f-ced1-48b5-a9ec-7ff6a2b3a9ee None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:18.950517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e3f9e28f-ced1-48b5-a9ec-7ff6a2b3a9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:19.017707 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9e28f-ced1-48b5-a9ec-7ff6a2b3a9ee None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:19.022211 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d2d07-c267-49b5-ba7d-27d7b410f9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:19.023081 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d2d07-c267-49b5-ba7d-27d7b410f9e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:19.023395 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d2d07-c267-49b5-ba7d-27d7b410f9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:19.023672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d2d07-c267-49b5-ba7d-27d7b410f9e0 admin admin] PATH_INFO: `/v3/users/62f33d76d51a4cc9a9f97f4ae2f5b0fe` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:19.024861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905d2d07-c267-49b5-ba7d-27d7b410f9e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62f33d76d51a4cc9a9f97f4ae2f5b0fe)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:19.026782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905d2d07-c267-49b5-ba7d-27d7b410f9e0 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:19.075438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-905d2d07-c267-49b5-ba7d-27d7b410f9e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62f33d76d51a4cc9a9f97f4ae2f5b0fe'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:19.087024 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-905d2d07-c267-49b5-ba7d-27d7b410f9e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62f33d76d51a4cc9a9f97f4ae2f5b0fe'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:19.093848 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-905d2d07-c267-49b5-ba7d-27d7b410f9e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62f33d76d51a4cc9a9f97f4ae2f5b0fe'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:19.095243 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-905d2d07-c267-49b5-ba7d-27d7b410f9e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62f33d76d51a4cc9a9f97f4ae2f5b0fe'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:19.102910 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-905d2d07-c267-49b5-ba7d-27d7b410f9e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62f33d76d51a4cc9a9f97f4ae2f5b0fe'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:19.109632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1048/2112] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:18 2022] DELETE /identity/v3/users/62f33d76d51a4cc9a9f97f4ae2f5b0fe => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:19.114621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2494c-0aa5-4e20-9cc1-9ff008a4168a None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:19.115147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2494c-0aa5-4e20-9cc1-9ff008a4168a None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:19.115259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2494c-0aa5-4e20-9cc1-9ff008a4168a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:19.132618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-68d2494c-0aa5-4e20-9cc1-9ff008a4168a None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:19.133887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-68d2494c-0aa5-4e20-9cc1-9ff008a4168a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:19.229037 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1065/2113] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:19.237267 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d2d07-c267-49b5-ba7d-27d7b410f9e0 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:19.239020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-905d2d07-c267-49b5-ba7d-27d7b410f9e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:19.321479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d2d07-c267-49b5-ba7d-27d7b410f9e0 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:19.325283 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca3c10-161a-479c-93c0-84788b3d4b6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:19.326018 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca3c10-161a-479c-93c0-84788b3d4b6e service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:19.326369 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca3c10-161a-479c-93c0-84788b3d4b6e service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:19.326712 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca3c10-161a-479c-93c0-84788b3d4b6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:19.337511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca3c10-161a-479c-93c0-84788b3d4b6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:19.338966 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca3c10-161a-479c-93c0-84788b3d4b6e service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:19.349625 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1049/2114] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:19.807537 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2494c-0aa5-4e20-9cc1-9ff008a4168a None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:19.809687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-68d2494c-0aa5-4e20-9cc1-9ff008a4168a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:19.837514 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2494c-0aa5-4e20-9cc1-9ff008a4168a None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:19.872923 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc2f7b-3d9a-45df-8693-4f82b2cb78f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:19.874021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc2f7b-3d9a-45df-8693-4f82b2cb78f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:19.874464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc2f7b-3d9a-45df-8693-4f82b2cb78f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:19.874667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc2f7b-3d9a-45df-8693-4f82b2cb78f2 admin admin] PATH_INFO: `/v3/projects/aed58fc9ac474ae39b24a9ef634fa6a5` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:19.876163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc2f7b-3d9a-45df-8693-4f82b2cb78f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aed58fc9ac474ae39b24a9ef634fa6a5)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:19.878138 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc2f7b-3d9a-45df-8693-4f82b2cb78f2 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:19.975023 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-27bc2f7b-3d9a-45df-8693-4f82b2cb78f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aed58fc9ac474ae39b24a9ef634fa6a5', 'request_id': 'req-27bc2f7b-3d9a-45df-8693-4f82b2cb78f2'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:19.982858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-27bc2f7b-3d9a-45df-8693-4f82b2cb78f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aed58fc9ac474ae39b24a9ef634fa6a5', 'request_id': 'req-27bc2f7b-3d9a-45df-8693-4f82b2cb78f2'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:19.991558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-27bc2f7b-3d9a-45df-8693-4f82b2cb78f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aed58fc9ac474ae39b24a9ef634fa6a5', 'request_id': 'req-27bc2f7b-3d9a-45df-8693-4f82b2cb78f2'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:20.011431 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-27bc2f7b-3d9a-45df-8693-4f82b2cb78f2 admin admin] The token cache is being invalidate because project aed58fc9ac474ae39b24a9ef634fa6a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:20.011726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-27bc2f7b-3d9a-45df-8693-4f82b2cb78f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:20.012374 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-27bc2f7b-3d9a-45df-8693-4f82b2cb78f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:20.014630 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1066/2115] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:19 2022] DELETE /identity/v3/projects/aed58fc9ac474ae39b24a9ef634fa6a5 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:20.022529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca3c10-161a-479c-93c0-84788b3d4b6e service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:20.034282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca3c10-161a-479c-93c0-84788b3d4b6e service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:20.044427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:20.098419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2abce-e1c0-4866-99c2-771f69b086e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:20.099677 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2abce-e1c0-4866-99c2-771f69b086e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:20.099993 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2abce-e1c0-4866-99c2-771f69b086e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:20.100282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2abce-e1c0-4866-99c2-771f69b086e4 admin admin] PATH_INFO: `/v3/users/7a3e43ed896a498fb9c6034e26075cac` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:20.101610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2abce-e1c0-4866-99c2-771f69b086e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a3e43ed896a498fb9c6034e26075cac)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:20.103631 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2abce-e1c0-4866-99c2-771f69b086e4 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:20.152346 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1ba2abce-e1c0-4866-99c2-771f69b086e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a3e43ed896a498fb9c6034e26075cac'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:20.166769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1ba2abce-e1c0-4866-99c2-771f69b086e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a3e43ed896a498fb9c6034e26075cac'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:20.175709 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1ba2abce-e1c0-4866-99c2-771f69b086e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a3e43ed896a498fb9c6034e26075cac'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:20.176738 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1ba2abce-e1c0-4866-99c2-771f69b086e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a3e43ed896a498fb9c6034e26075cac'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:20.184666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1ba2abce-e1c0-4866-99c2-771f69b086e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a3e43ed896a498fb9c6034e26075cac'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:20.191539 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1050/2116] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:20 2022] DELETE /identity/v3/users/7a3e43ed896a498fb9c6034e26075cac => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:20.498360 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc2f7b-3d9a-45df-8693-4f82b2cb78f2 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:20.501711 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-27bc2f7b-3d9a-45df-8693-4f82b2cb78f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:20.524988 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc2f7b-3d9a-45df-8693-4f82b2cb78f2 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:20.626584 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982eef5-8081-4e47-b325-54f3bfeb3015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:20.627889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982eef5-8081-4e47-b325-54f3bfeb3015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:20.628479 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982eef5-8081-4e47-b325-54f3bfeb3015 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:20.628949 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982eef5-8081-4e47-b325-54f3bfeb3015 admin admin] PATH_INFO: `/v3/projects/07fce36833d747e6b56108c85daf4143` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:20.631709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982eef5-8081-4e47-b325-54f3bfeb3015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07fce36833d747e6b56108c85daf4143)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:20.634136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982eef5-8081-4e47-b325-54f3bfeb3015 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:20.660178 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:20.707072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c982eef5-8081-4e47-b325-54f3bfeb3015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07fce36833d747e6b56108c85daf4143', 'request_id': 'req-c982eef5-8081-4e47-b325-54f3bfeb3015'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:20.716453 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c982eef5-8081-4e47-b325-54f3bfeb3015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07fce36833d747e6b56108c85daf4143', 'request_id': 'req-c982eef5-8081-4e47-b325-54f3bfeb3015'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:20.727865 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c982eef5-8081-4e47-b325-54f3bfeb3015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07fce36833d747e6b56108c85daf4143', 'request_id': 'req-c982eef5-8081-4e47-b325-54f3bfeb3015'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:20.744030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c982eef5-8081-4e47-b325-54f3bfeb3015 admin admin] The token cache is being invalidate because project 07fce36833d747e6b56108c85daf4143 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:20.744030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c982eef5-8081-4e47-b325-54f3bfeb3015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:20.745081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c982eef5-8081-4e47-b325-54f3bfeb3015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:20.749933 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1067/2117] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:20 2022] DELETE /identity/v3/projects/07fce36833d747e6b56108c85daf4143 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:20.751534 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2abce-e1c0-4866-99c2-771f69b086e4 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:20.754693 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-1ba2abce-e1c0-4866-99c2-771f69b086e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:20.792212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2abce-e1c0-4866-99c2-771f69b086e4 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:20.933363 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982eef5-8081-4e47-b325-54f3bfeb3015 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:20.936709 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e47872-5c08-4d34-9873-c361b83edd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:20.937824 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e47872-5c08-4d34-9873-c361b83edd0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:20.938038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-c982eef5-8081-4e47-b325-54f3bfeb3015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:20.938668 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e47872-5c08-4d34-9873-c361b83edd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:20.938979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e47872-5c08-4d34-9873-c361b83edd0c admin admin] PATH_INFO: `/v3/users/0178361d94c74741903e243e6fee2a32` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:20.940531 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e47872-5c08-4d34-9873-c361b83edd0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0178361d94c74741903e243e6fee2a32)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:20.942581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e47872-5c08-4d34-9873-c361b83edd0c admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:21.021558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982eef5-8081-4e47-b325-54f3bfeb3015 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:21.033716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:21.049668 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-54e47872-5c08-4d34-9873-c361b83edd0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0178361d94c74741903e243e6fee2a32'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:21.067469 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bedb60-9d4d-4239-9046-a28e269fb777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:21.073263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bedb60-9d4d-4239-9046-a28e269fb777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:21.073595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bedb60-9d4d-4239-9046-a28e269fb777 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:21.073832 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bedb60-9d4d-4239-9046-a28e269fb777 admin admin] PATH_INFO: `/v3/users/dbbcf057a4744909bea506f6a7da003b` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:21.078905 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-54e47872-5c08-4d34-9873-c361b83edd0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0178361d94c74741903e243e6fee2a32'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:21.082818 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bedb60-9d4d-4239-9046-a28e269fb777 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbbcf057a4744909bea506f6a7da003b)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:21.090756 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bedb60-9d4d-4239-9046-a28e269fb777 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:21.111071 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-54e47872-5c08-4d34-9873-c361b83edd0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0178361d94c74741903e243e6fee2a32'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:21.111850 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-54e47872-5c08-4d34-9873-c361b83edd0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0178361d94c74741903e243e6fee2a32'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:21.131991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-54e47872-5c08-4d34-9873-c361b83edd0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0178361d94c74741903e243e6fee2a32'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:21.147222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1051/2118] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:20 2022] DELETE /identity/v3/users/0178361d94c74741903e243e6fee2a32 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:21.184544 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-27bedb60-9d4d-4239-9046-a28e269fb777 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbbcf057a4744909bea506f6a7da003b'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:21.199440 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-27bedb60-9d4d-4239-9046-a28e269fb777 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbbcf057a4744909bea506f6a7da003b'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:21.209398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-27bedb60-9d4d-4239-9046-a28e269fb777 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbbcf057a4744909bea506f6a7da003b'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:21.218073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-27bedb60-9d4d-4239-9046-a28e269fb777 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbbcf057a4744909bea506f6a7da003b'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:21.218736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-27bedb60-9d4d-4239-9046-a28e269fb777 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbbcf057a4744909bea506f6a7da003b'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:21.226729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1068/2119] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:20 2022] DELETE /identity/v3/users/dbbcf057a4744909bea506f6a7da003b => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:21.231233 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d54f9b-bf22-4730-b286-0f1c629aeed9 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:21.231556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d54f9b-bf22-4730-b286-0f1c629aeed9 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:21.231836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d54f9b-bf22-4730-b286-0f1c629aeed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:21.257749 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-e9d54f9b-bf22-4730-b286-0f1c629aeed9 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:21.259470 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e9d54f9b-bf22-4730-b286-0f1c629aeed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:21.340987 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1052/2120] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:21.351328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bedb60-9d4d-4239-9046-a28e269fb777 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:21.352889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-27bedb60-9d4d-4239-9046-a28e269fb777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:21.457491 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bedb60-9d4d-4239-9046-a28e269fb777 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:21.471047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc3276-5bf0-487b-8987-6db3e6da62c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:21.471885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc3276-5bf0-487b-8987-6db3e6da62c6 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:21.473544 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc3276-5bf0-487b-8987-6db3e6da62c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:21.473789 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc3276-5bf0-487b-8987-6db3e6da62c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:21.486657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc3276-5bf0-487b-8987-6db3e6da62c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:21.488646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc3276-5bf0-487b-8987-6db3e6da62c6 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:21.500806 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1069/2121] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:22.185554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d54f9b-bf22-4730-b286-0f1c629aeed9 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:22.190028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc3276-5bf0-487b-8987-6db3e6da62c6 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:22.190383 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e9d54f9b-bf22-4730-b286-0f1c629aeed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:22.191612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-7fbc3276-5bf0-487b-8987-6db3e6da62c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:22.208092 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d54f9b-bf22-4730-b286-0f1c629aeed9 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:22.211137 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc3276-5bf0-487b-8987-6db3e6da62c6 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:22.214705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef642a-cbf3-48fd-81ac-90304092de6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:22.216127 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef642a-cbf3-48fd-81ac-90304092de6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:22.216873 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef642a-cbf3-48fd-81ac-90304092de6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:22.217386 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef642a-cbf3-48fd-81ac-90304092de6f admin admin] PATH_INFO: `/v3/projects/0a91f4b2bf5e49adb47b96bf2d99af1b` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:22.218820 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef642a-cbf3-48fd-81ac-90304092de6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a91f4b2bf5e49adb47b96bf2d99af1b)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:22.221381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef642a-cbf3-48fd-81ac-90304092de6f admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:22.227041 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a41cd-7df2-4615-b037-fdd46476a205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:22.227950 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a41cd-7df2-4615-b037-fdd46476a205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:22.233014 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a41cd-7df2-4615-b037-fdd46476a205 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:22.233309 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a41cd-7df2-4615-b037-fdd46476a205 admin admin] PATH_INFO: `/v3/projects/b9a6fd2569d640b0b64a3f9e07ceaebd` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:22.237450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a41cd-7df2-4615-b037-fdd46476a205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9a6fd2569d640b0b64a3f9e07ceaebd)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:22.239647 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a41cd-7df2-4615-b037-fdd46476a205 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:22.282480 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:22.354518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f42a41cd-7df2-4615-b037-fdd46476a205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9a6fd2569d640b0b64a3f9e07ceaebd', 'request_id': 'req-f42a41cd-7df2-4615-b037-fdd46476a205'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:22.370469 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-88ef642a-cbf3-48fd-81ac-90304092de6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a91f4b2bf5e49adb47b96bf2d99af1b', 'request_id': 'req-88ef642a-cbf3-48fd-81ac-90304092de6f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:22.373142 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f42a41cd-7df2-4615-b037-fdd46476a205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9a6fd2569d640b0b64a3f9e07ceaebd', 'request_id': 'req-f42a41cd-7df2-4615-b037-fdd46476a205'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:22.384635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-88ef642a-cbf3-48fd-81ac-90304092de6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a91f4b2bf5e49adb47b96bf2d99af1b', 'request_id': 'req-88ef642a-cbf3-48fd-81ac-90304092de6f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:22.394308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-88ef642a-cbf3-48fd-81ac-90304092de6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a91f4b2bf5e49adb47b96bf2d99af1b', 'request_id': 'req-88ef642a-cbf3-48fd-81ac-90304092de6f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:22.398722 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f42a41cd-7df2-4615-b037-fdd46476a205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9a6fd2569d640b0b64a3f9e07ceaebd', 'request_id': 'req-f42a41cd-7df2-4615-b037-fdd46476a205'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:22.434174 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f42a41cd-7df2-4615-b037-fdd46476a205 admin admin] The token cache is being invalidate because project b9a6fd2569d640b0b64a3f9e07ceaebd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:22.434482 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f42a41cd-7df2-4615-b037-fdd46476a205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:22.436240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f42a41cd-7df2-4615-b037-fdd46476a205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:22.440425 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-88ef642a-cbf3-48fd-81ac-90304092de6f admin admin] The token cache is being invalidate because project 0a91f4b2bf5e49adb47b96bf2d99af1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:22.440721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-88ef642a-cbf3-48fd-81ac-90304092de6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:22.445822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1070/2122] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:22 2022] DELETE /identity/v3/projects/b9a6fd2569d640b0b64a3f9e07ceaebd => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:22.446037 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-88ef642a-cbf3-48fd-81ac-90304092de6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:22.448543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719703b-3640-4c03-b816-ec0ea9f80bfe None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:22.448820 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1053/2123] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:22 2022] DELETE /identity/v3/projects/0a91f4b2bf5e49adb47b96bf2d99af1b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:22.448909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719703b-3640-4c03-b816-ec0ea9f80bfe None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:22.449151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719703b-3640-4c03-b816-ec0ea9f80bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:22.453644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef642a-cbf3-48fd-81ac-90304092de6f admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:22.457303 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-88ef642a-cbf3-48fd-81ac-90304092de6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:22.516883 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-3719703b-3640-4c03-b816-ec0ea9f80bfe None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:22.518390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-3719703b-3640-4c03-b816-ec0ea9f80bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:22.577161 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:22.671273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef642a-cbf3-48fd-81ac-90304092de6f admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:22.688095 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1071/2124] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:22.689959 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9510c1-05cb-474a-9642-d3da1c270f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:22.694654 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9510c1-05cb-474a-9642-d3da1c270f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:22.694949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9510c1-05cb-474a-9642-d3da1c270f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:22.695248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9510c1-05cb-474a-9642-d3da1c270f9e admin admin] PATH_INFO: `/v3/users/2673b3a107854e63959bb4e8dbaae2f6` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:22.697231 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ec87e-78a3-428a-b156-1547ebbb6407 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:22.697527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ec87e-78a3-428a-b156-1547ebbb6407 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:22.697809 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ec87e-78a3-428a-b156-1547ebbb6407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:22.702293 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9510c1-05cb-474a-9642-d3da1c270f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2673b3a107854e63959bb4e8dbaae2f6)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:22.705117 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9510c1-05cb-474a-9642-d3da1c270f9e admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:22.738373 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-bd1ec87e-78a3-428a-b156-1547ebbb6407 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:22.739643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-bd1ec87e-78a3-428a-b156-1547ebbb6407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:22.767402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1072/2125] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:22.768335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ec87e-78a3-428a-b156-1547ebbb6407 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:22.806105 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ec87e-78a3-428a-b156-1547ebbb6407 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:22.812953 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ffa05-e787-4c04-a46b-2d9911d12aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:22.813832 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ffa05-e787-4c04-a46b-2d9911d12aff admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:22.814009 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ffa05-e787-4c04-a46b-2d9911d12aff admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:22.814432 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049ffa05-e787-4c04-a46b-2d9911d12aff admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:22.815161 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ffa05-e787-4c04-a46b-2d9911d12aff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:46:22.815516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ffa05-e787-4c04-a46b-2d9911d12aff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:22.817121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ab9510c1-05cb-474a-9642-d3da1c270f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2673b3a107854e63959bb4e8dbaae2f6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:22.817326 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049ffa05-e787-4c04-a46b-2d9911d12aff admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:22.831996 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1073/2126] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:46:22.844982 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ffa05-e787-4c04-a46b-2d9911d12aff admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:22.849012 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ab9510c1-05cb-474a-9642-d3da1c270f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2673b3a107854e63959bb4e8dbaae2f6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:22.859524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ab9510c1-05cb-474a-9642-d3da1c270f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2673b3a107854e63959bb4e8dbaae2f6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:22.861731 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ab9510c1-05cb-474a-9642-d3da1c270f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2673b3a107854e63959bb4e8dbaae2f6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:22.870640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049ffa05-e787-4c04-a46b-2d9911d12aff admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:22.877291 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-ab9510c1-05cb-474a-9642-d3da1c270f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2673b3a107854e63959bb4e8dbaae2f6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:22.882636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec774dfd-14ba-42a4-b4b8-fdaf1e0b6477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:22.883318 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec774dfd-14ba-42a4-b4b8-fdaf1e0b6477 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:22.883627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec774dfd-14ba-42a4-b4b8-fdaf1e0b6477 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:22.884568 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec774dfd-14ba-42a4-b4b8-fdaf1e0b6477 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:22.885278 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec774dfd-14ba-42a4-b4b8-fdaf1e0b6477 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:22.887292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec774dfd-14ba-42a4-b4b8-fdaf1e0b6477 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:22.889206 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1054/2127] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:22 2022] DELETE /identity/v3/users/2673b3a107854e63959bb4e8dbaae2f6 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:22.952327 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1074/2128] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:22 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:22.954463 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9510c1-05cb-474a-9642-d3da1c270f9e admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:22.956364 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ab9510c1-05cb-474a-9642-d3da1c270f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:23.058595 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9510c1-05cb-474a-9642-d3da1c270f9e admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:23.178713 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37025d4a-a153-4b82-bd78-98e794225c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:23.179490 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37025d4a-a153-4b82-bd78-98e794225c74 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:23.179787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37025d4a-a153-4b82-bd78-98e794225c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:23.180782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37025d4a-a153-4b82-bd78-98e794225c74 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:23.181634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37025d4a-a153-4b82-bd78-98e794225c74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:23.183979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37025d4a-a153-4b82-bd78-98e794225c74 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:23.196664 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec774dfd-14ba-42a4-b4b8-fdaf1e0b6477 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:23.198498 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ec774dfd-14ba-42a4-b4b8-fdaf1e0b6477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:23.220372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:23.225574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec774dfd-14ba-42a4-b4b8-fdaf1e0b6477 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:23.242359 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9819acd-7af3-4d40-ae7f-c9773ca0ad05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:23.245348 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1055/2129] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:22 2022] POST /identity/v3/users => generated 284 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:23.246159 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9819acd-7af3-4d40-ae7f-c9773ca0ad05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:23.246664 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9819acd-7af3-4d40-ae7f-c9773ca0ad05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:23.247154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9819acd-7af3-4d40-ae7f-c9773ca0ad05 admin admin] PATH_INFO: `/v3/users/4465844651ec4ddaa6a772b1aca23db2` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:23.248763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9819acd-7af3-4d40-ae7f-c9773ca0ad05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4465844651ec4ddaa6a772b1aca23db2)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:23.249926 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37025d4a-a153-4b82-bd78-98e794225c74 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:23.253467 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9819acd-7af3-4d40-ae7f-c9773ca0ad05 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:23.266847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37025d4a-a153-4b82-bd78-98e794225c74 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:23.273122 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1aea6-b595-4529-91e4-482a7d39e66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:23.274505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1aea6-b595-4529-91e4-482a7d39e66e admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:23.275792 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1aea6-b595-4529-91e4-482a7d39e66e admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:23.276483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1aea6-b595-4529-91e4-482a7d39e66e admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:23.277488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1aea6-b595-4529-91e4-482a7d39e66e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:23.279839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1aea6-b595-4529-91e4-482a7d39e66e admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:23.293590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1056/2130] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:23.297805 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1aea6-b595-4529-91e4-482a7d39e66e admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:23.316672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1aea6-b595-4529-91e4-482a7d39e66e admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:23.328071 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfea95d-fc43-4cbf-8557-2976daa4f18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:23.330079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfea95d-fc43-4cbf-8557-2976daa4f18b admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:23.331384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfea95d-fc43-4cbf-8557-2976daa4f18b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:23.332009 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfea95d-fc43-4cbf-8557-2976daa4f18b admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:23.333492 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfea95d-fc43-4cbf-8557-2976daa4f18b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:23.336101 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-a9819acd-7af3-4d40-ae7f-c9773ca0ad05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4465844651ec4ddaa6a772b1aca23db2'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:23.336629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfea95d-fc43-4cbf-8557-2976daa4f18b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:23.350159 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:23.355912 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1057/2131] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:23.360300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfea95d-fc43-4cbf-8557-2976daa4f18b admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:23.364320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-a9819acd-7af3-4d40-ae7f-c9773ca0ad05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4465844651ec4ddaa6a772b1aca23db2'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:23.376387 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfea95d-fc43-4cbf-8557-2976daa4f18b admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:23.383741 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc91e2-8f74-4f76-a40f-b3498bd91b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:23.385049 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc91e2-8f74-4f76-a40f-b3498bd91b21 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:23.385548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc91e2-8f74-4f76-a40f-b3498bd91b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:23.386146 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc91e2-8f74-4f76-a40f-b3498bd91b21 admin admin] PATH_INFO: `/v3/projects/8924f21d5e90417393b32c64dd1fbb81/users/b6c69ec246b442d0b8be70425254a42f/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:23.396585 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-a9819acd-7af3-4d40-ae7f-c9773ca0ad05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4465844651ec4ddaa6a772b1aca23db2'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:23.409889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-a9819acd-7af3-4d40-ae7f-c9773ca0ad05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4465844651ec4ddaa6a772b1aca23db2'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:23.410792 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-a9819acd-7af3-4d40-ae7f-c9773ca0ad05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4465844651ec4ddaa6a772b1aca23db2'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:23.424517 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1075/2132] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:23 2022] DELETE /identity/v3/users/4465844651ec4ddaa6a772b1aca23db2 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:23.433233 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc91e2-8f74-4f76-a40f-b3498bd91b21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8924f21d5e90417393b32c64dd1fbb81, user_id=b6c69ec246b442d0b8be70425254a42f, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:23.433706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b25622-bcc6-43d7-b28d-3fc2cb99999f None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:23.434074 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b25622-bcc6-43d7-b28d-3fc2cb99999f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:23.434355 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b25622-bcc6-43d7-b28d-3fc2cb99999f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:23.442700 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc91e2-8f74-4f76-a40f-b3498bd91b21 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:23.488618 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1058/2133] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:23 2022] PUT /identity/v3/projects/8924f21d5e90417393b32c64dd1fbb81/users/b6c69ec246b442d0b8be70425254a42f/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:23.499277 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-82b25622-bcc6-43d7-b28d-3fc2cb99999f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:23.499277 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-82b25622-bcc6-43d7-b28d-3fc2cb99999f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:23.501427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe337f2d-3e38-4240-9aa6-ac23b71549d7 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:23.501881 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe337f2d-3e38-4240-9aa6-ac23b71549d7 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:23.502180 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe337f2d-3e38-4240-9aa6-ac23b71549d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:23.549569 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-fe337f2d-3e38-4240-9aa6-ac23b71549d7 None None] MFA Rules not processed for user `b6c69ec246b442d0b8be70425254a42f`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:23.556156 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-fe337f2d-3e38-4240-9aa6-ac23b71549d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:23.626715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1076/2134] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:23.674409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b25622-bcc6-43d7-b28d-3fc2cb99999f None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:23.678524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-82b25622-bcc6-43d7-b28d-3fc2cb99999f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:23.765934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b25622-bcc6-43d7-b28d-3fc2cb99999f None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:23.782599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdbfb3-22b9-4ea0-877a-fe3c54f22bd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:23.783994 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdbfb3-22b9-4ea0-877a-fe3c54f22bd4 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:23.784502 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdbfb3-22b9-4ea0-877a-fe3c54f22bd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:23.785087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdbfb3-22b9-4ea0-877a-fe3c54f22bd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:23.801837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1059/2135] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:23 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:23.801837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe337f2d-3e38-4240-9aa6-ac23b71549d7 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:23.808890 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbdbfb3-22b9-4ea0-877a-fe3c54f22bd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:23.809388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-fe337f2d-3e38-4240-9aa6-ac23b71549d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:23.811181 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbdbfb3-22b9-4ea0-877a-fe3c54f22bd4 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:23.831897 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe337f2d-3e38-4240-9aa6-ac23b71549d7 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:23.832457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1077/2136] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:23.833484 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdbfb3-22b9-4ea0-877a-fe3c54f22bd4 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:23.834866 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-0bbdbfb3-22b9-4ea0-877a-fe3c54f22bd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:23.857669 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62869c28-bdb8-4c23-bc68-8d3de75c58cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:23.858648 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62869c28-bdb8-4c23-bc68-8d3de75c58cb admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:23.858941 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62869c28-bdb8-4c23-bc68-8d3de75c58cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:23.863038 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62869c28-bdb8-4c23-bc68-8d3de75c58cb admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:23.863994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62869c28-bdb8-4c23-bc68-8d3de75c58cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:23.870903 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62869c28-bdb8-4c23-bc68-8d3de75c58cb admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:23.901586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdbfb3-22b9-4ea0-877a-fe3c54f22bd4 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:23.905688 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db2e08-9a78-4889-ade9-06807ce9cff1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:23.906360 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db2e08-9a78-4889-ade9-06807ce9cff1 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:23.906739 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db2e08-9a78-4889-ade9-06807ce9cff1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:23.907026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db2e08-9a78-4889-ade9-06807ce9cff1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:23.910144 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-57db2e08-9a78-4889-ade9-06807ce9cff1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:23.956662 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1060/2137] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:23 2022] POST /identity/v3/projects => generated 365 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:23.961289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62869c28-bdb8-4c23-bc68-8d3de75c58cb admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:23.970611 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:24.047162 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db2e08-9a78-4889-ade9-06807ce9cff1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:24.049353 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db2e08-9a78-4889-ade9-06807ce9cff1 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:24.077022 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1078/2138] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:24.137545 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62869c28-bdb8-4c23-bc68-8d3de75c58cb admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:24.151987 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36e95c-b6ca-4bf5-a9d7-81f739dcc677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:24.152685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36e95c-b6ca-4bf5-a9d7-81f739dcc677 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:24.152968 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36e95c-b6ca-4bf5-a9d7-81f739dcc677 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:24.153246 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36e95c-b6ca-4bf5-a9d7-81f739dcc677 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:24.153948 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d36e95c-b6ca-4bf5-a9d7-81f739dcc677 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:24.155914 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d36e95c-b6ca-4bf5-a9d7-81f739dcc677 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:24.203191 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1061/2139] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:23 2022] POST /identity/v3/users => generated 289 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:24.207123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db2e08-9a78-4889-ade9-06807ce9cff1 service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:24.226552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db2e08-9a78-4889-ade9-06807ce9cff1 service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:24.249875 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf0b12-be84-4706-a404-3a0de5e91e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:24.250478 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf0b12-be84-4706-a404-3a0de5e91e27 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:24.250765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf0b12-be84-4706-a404-3a0de5e91e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:24.251587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf0b12-be84-4706-a404-3a0de5e91e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:24.252485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf0b12-be84-4706-a404-3a0de5e91e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:24.259216 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaf0b12-be84-4706-a404-3a0de5e91e27 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:24.302112 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1079/2140] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:46:24.303431 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf0b12-be84-4706-a404-3a0de5e91e27 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:24.303989 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36e95c-b6ca-4bf5-a9d7-81f739dcc677 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:24.308569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-2aaf0b12-be84-4706-a404-3a0de5e91e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:24.310434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:24.324586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36e95c-b6ca-4bf5-a9d7-81f739dcc677 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:24.332450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100af0e5-6adf-4458-99e1-e1db9dc4e779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:24.335111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf0b12-be84-4706-a404-3a0de5e91e27 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:24.341247 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100af0e5-6adf-4458-99e1-e1db9dc4e779 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:24.341941 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100af0e5-6adf-4458-99e1-e1db9dc4e779 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:24.342920 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100af0e5-6adf-4458-99e1-e1db9dc4e779 admin admin] PATH_INFO: `/v3/projects/e8da91a7e1f3489e979e4a14154ed054/users/01885b8efaff48ce8ca3c5ce58a8d955/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:24.349136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c58c2c-dd2c-4fd7-972c-be8266422ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:24.350133 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c58c2c-dd2c-4fd7-972c-be8266422ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:24.350424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c58c2c-dd2c-4fd7-972c-be8266422ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:24.350707 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c58c2c-dd2c-4fd7-972c-be8266422ccd admin admin] PATH_INFO: `/v3/projects/24f8a5d6bc38496792b1fa1562a159ab` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:24.355339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c58c2c-dd2c-4fd7-972c-be8266422ccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=24f8a5d6bc38496792b1fa1562a159ab)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:24.359336 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c58c2c-dd2c-4fd7-972c-be8266422ccd admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:24.406189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100af0e5-6adf-4458-99e1-e1db9dc4e779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8da91a7e1f3489e979e4a14154ed054, user_id=01885b8efaff48ce8ca3c5ce58a8d955, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:24.409973 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100af0e5-6adf-4458-99e1-e1db9dc4e779 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:24.440570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1062/2141] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:24 2022] PUT /identity/v3/projects/e8da91a7e1f3489e979e4a14154ed054/users/01885b8efaff48ce8ca3c5ce58a8d955/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:24.444210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100af0e5-6adf-4458-99e1-e1db9dc4e779 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:24.466044 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c9c58c2c-dd2c-4fd7-972c-be8266422ccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24f8a5d6bc38496792b1fa1562a159ab', 'request_id': 'req-c9c58c2c-dd2c-4fd7-972c-be8266422ccd'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:24.475102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c9c58c2c-dd2c-4fd7-972c-be8266422ccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24f8a5d6bc38496792b1fa1562a159ab', 'request_id': 'req-c9c58c2c-dd2c-4fd7-972c-be8266422ccd'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:24.486498 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c9c58c2c-dd2c-4fd7-972c-be8266422ccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24f8a5d6bc38496792b1fa1562a159ab', 'request_id': 'req-c9c58c2c-dd2c-4fd7-972c-be8266422ccd'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:24.520044 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c9c58c2c-dd2c-4fd7-972c-be8266422ccd admin admin] The token cache is being invalidate because project 24f8a5d6bc38496792b1fa1562a159ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:24.520369 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c9c58c2c-dd2c-4fd7-972c-be8266422ccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:24.522069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c9c58c2c-dd2c-4fd7-972c-be8266422ccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:24.527912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1080/2142] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:24 2022] DELETE /identity/v3/projects/24f8a5d6bc38496792b1fa1562a159ab => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:24.533842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c58c2c-dd2c-4fd7-972c-be8266422ccd admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:24.536938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-c9c58c2c-dd2c-4fd7-972c-be8266422ccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:24.612521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100af0e5-6adf-4458-99e1-e1db9dc4e779 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:24.694235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c58c2c-dd2c-4fd7-972c-be8266422ccd admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:24.709967 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdeb840-5da6-4de6-8ad8-dda7fa3ef449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:24.713444 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeb840-5da6-4de6-8ad8-dda7fa3ef449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:24.714691 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeb840-5da6-4de6-8ad8-dda7fa3ef449 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:24.714691 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeb840-5da6-4de6-8ad8-dda7fa3ef449 admin admin] PATH_INFO: `/v3/users/df50a7bd2c3345afa62265df051b8f2f` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:24.716930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdeb840-5da6-4de6-8ad8-dda7fa3ef449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df50a7bd2c3345afa62265df051b8f2f)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:24.728115 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdeb840-5da6-4de6-8ad8-dda7fa3ef449 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:24.762347 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5237dc-d2d4-48fc-8432-2dd75ca044d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:24.763027 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5237dc-d2d4-48fc-8432-2dd75ca044d0 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:24.763306 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5237dc-d2d4-48fc-8432-2dd75ca044d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:24.763624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5237dc-d2d4-48fc-8432-2dd75ca044d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:24.764369 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5237dc-d2d4-48fc-8432-2dd75ca044d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:24.768431 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5237dc-d2d4-48fc-8432-2dd75ca044d0 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:24.814164 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1063/2143] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:24.815532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5237dc-d2d4-48fc-8432-2dd75ca044d0 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:24.817917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-dc5237dc-d2d4-48fc-8432-2dd75ca044d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:24.839173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:24.851286 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-ecdeb840-5da6-4de6-8ad8-dda7fa3ef449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df50a7bd2c3345afa62265df051b8f2f'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:24.861779 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5237dc-d2d4-48fc-8432-2dd75ca044d0 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:24.869105 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccaf33d-5a2f-432e-8b00-e16f49dbe304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:24.870384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccaf33d-5a2f-432e-8b00-e16f49dbe304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:24.871333 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccaf33d-5a2f-432e-8b00-e16f49dbe304 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:24.872104 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccaf33d-5a2f-432e-8b00-e16f49dbe304 admin admin] PATH_INFO: `/v3/projects/4e4d9018302f48f7bd90d21deb3b9895` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:24.873756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccaf33d-5a2f-432e-8b00-e16f49dbe304 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e4d9018302f48f7bd90d21deb3b9895)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:24.876732 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccaf33d-5a2f-432e-8b00-e16f49dbe304 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:24.889743 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-ecdeb840-5da6-4de6-8ad8-dda7fa3ef449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df50a7bd2c3345afa62265df051b8f2f'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:24.908021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-ecdeb840-5da6-4de6-8ad8-dda7fa3ef449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df50a7bd2c3345afa62265df051b8f2f'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:24.929704 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-ecdeb840-5da6-4de6-8ad8-dda7fa3ef449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df50a7bd2c3345afa62265df051b8f2f'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:24.930823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-ecdeb840-5da6-4de6-8ad8-dda7fa3ef449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df50a7bd2c3345afa62265df051b8f2f'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:24.945052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1081/2144] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:24 2022] DELETE /identity/v3/users/df50a7bd2c3345afa62265df051b8f2f => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:24.946152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdeb840-5da6-4de6-8ad8-dda7fa3ef449 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:24.947820 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ecdeb840-5da6-4de6-8ad8-dda7fa3ef449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:24.964923 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9ccaf33d-5a2f-432e-8b00-e16f49dbe304 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e4d9018302f48f7bd90d21deb3b9895', 'request_id': 'req-9ccaf33d-5a2f-432e-8b00-e16f49dbe304'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:24.978725 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9ccaf33d-5a2f-432e-8b00-e16f49dbe304 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e4d9018302f48f7bd90d21deb3b9895', 'request_id': 'req-9ccaf33d-5a2f-432e-8b00-e16f49dbe304'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:24.989258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9ccaf33d-5a2f-432e-8b00-e16f49dbe304 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e4d9018302f48f7bd90d21deb3b9895', 'request_id': 'req-9ccaf33d-5a2f-432e-8b00-e16f49dbe304'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:25.002810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9ccaf33d-5a2f-432e-8b00-e16f49dbe304 admin admin] The token cache is being invalidate because project 4e4d9018302f48f7bd90d21deb3b9895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:25.003489 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9ccaf33d-5a2f-432e-8b00-e16f49dbe304 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:25.004382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-9ccaf33d-5a2f-432e-8b00-e16f49dbe304 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:25.006695 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1064/2145] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:24 2022] DELETE /identity/v3/projects/4e4d9018302f48f7bd90d21deb3b9895 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:25.017708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15aaf0c-c201-4ddc-b013-d4276bdc3f09 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:25.019179 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15aaf0c-c201-4ddc-b013-d4276bdc3f09 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:25.019727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15aaf0c-c201-4ddc-b013-d4276bdc3f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:25.052847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:25.082676 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-f15aaf0c-c201-4ddc-b013-d4276bdc3f09 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:25.086305 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f15aaf0c-c201-4ddc-b013-d4276bdc3f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:25.104900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdeb840-5da6-4de6-8ad8-dda7fa3ef449 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:25.111405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1065/2146] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:25.127362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff63087-154b-41fb-9268-7fce028b9bf4 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:25.128864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff63087-154b-41fb-9268-7fce028b9bf4 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:25.129303 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff63087-154b-41fb-9268-7fce028b9bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:25.130532 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d7531-87a3-4456-8f2a-9cc25e6bcdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:25.131552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d7531-87a3-4456-8f2a-9cc25e6bcdc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:25.131881 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d7531-87a3-4456-8f2a-9cc25e6bcdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:25.132168 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d7531-87a3-4456-8f2a-9cc25e6bcdc6 admin admin] PATH_INFO: `/v3/projects/e8da91a7e1f3489e979e4a14154ed054/users/01885b8efaff48ce8ca3c5ce58a8d955/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:25.137782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d7531-87a3-4456-8f2a-9cc25e6bcdc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8da91a7e1f3489e979e4a14154ed054, user_id=01885b8efaff48ce8ca3c5ce58a8d955, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:25.142572 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d7531-87a3-4456-8f2a-9cc25e6bcdc6 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:25.172947 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-0ff63087-154b-41fb-9268-7fce028b9bf4 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:25.175718 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-0ff63087-154b-41fb-9268-7fce028b9bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:25.192120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1082/2147] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:24 2022] PUT /identity/v3/projects/e8da91a7e1f3489e979e4a14154ed054/users/01885b8efaff48ce8ca3c5ce58a8d955/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:25.197426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b7255-5669-4fd8-ac2d-9dcf421fd59c None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:25.197894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b7255-5669-4fd8-ac2d-9dcf421fd59c None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:25.198326 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b7255-5669-4fd8-ac2d-9dcf421fd59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:25.274747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-093b7255-5669-4fd8-ac2d-9dcf421fd59c None None] MFA Rules not processed for user `01885b8efaff48ce8ca3c5ce58a8d955`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:25.289308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-093b7255-5669-4fd8-ac2d-9dcf421fd59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:25.361916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1066/2148] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:25.367000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff63087-154b-41fb-9268-7fce028b9bf4 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:25.383618 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff63087-154b-41fb-9268-7fce028b9bf4 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:25.393716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0f89f-b045-4f10-aebc-03d0ebea449a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:25.393716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0f89f-b045-4f10-aebc-03d0ebea449a admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:25.393716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0f89f-b045-4f10-aebc-03d0ebea449a admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:25.393716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0f89f-b045-4f10-aebc-03d0ebea449a admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:25.395097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0f89f-b045-4f10-aebc-03d0ebea449a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:46:25.395667 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0f89f-b045-4f10-aebc-03d0ebea449a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:25.398623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0f89f-b045-4f10-aebc-03d0ebea449a admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:25.426262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1067/2149] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:46:25.431061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0f89f-b045-4f10-aebc-03d0ebea449a admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:25.453199 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0f89f-b045-4f10-aebc-03d0ebea449a admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:25.464368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6d89e-d364-495d-a90b-1f8f2a7533c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:25.465103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6d89e-d364-495d-a90b-1f8f2a7533c1 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:25.465587 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6d89e-d364-495d-a90b-1f8f2a7533c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:25.465884 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6d89e-d364-495d-a90b-1f8f2a7533c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:25.466601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6d89e-d364-495d-a90b-1f8f2a7533c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:25.469521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6d89e-d364-495d-a90b-1f8f2a7533c1 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:25.481987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1083/2150] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:25 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:25.492465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b7255-5669-4fd8-ac2d-9dcf421fd59c None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:25.494088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-093b7255-5669-4fd8-ac2d-9dcf421fd59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:25.513649 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1068/2151] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:25 2022] POST /identity/v3/projects => generated 375 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:46:25.517419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6d89e-d364-495d-a90b-1f8f2a7533c1 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:25.565652 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:25.603863 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b7255-5669-4fd8-ac2d-9dcf421fd59c None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:25.652306 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6d89e-d364-495d-a90b-1f8f2a7533c1 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:25.671168 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c2108-02b0-4bfe-8e64-d0f277c53eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:25.672954 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c2108-02b0-4bfe-8e64-d0f277c53eb1 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:25.672954 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c2108-02b0-4bfe-8e64-d0f277c53eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:25.673193 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c2108-02b0-4bfe-8e64-d0f277c53eb1 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:25.674026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c2108-02b0-4bfe-8e64-d0f277c53eb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:25.676189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c2108-02b0-4bfe-8e64-d0f277c53eb1 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:25.706435 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b0ac0-06a0-4f90-a965-e1f1b0609fed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:25.707693 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b0ac0-06a0-4f90-a965-e1f1b0609fed service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:25.708221 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b0ac0-06a0-4f90-a965-e1f1b0609fed service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:25.708497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1069/2152] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:25 2022] POST /identity/v3/users => generated 288 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:46:25.708896 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b0ac0-06a0-4f90-a965-e1f1b0609fed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:25.709471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c2108-02b0-4bfe-8e64-d0f277c53eb1 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:25.717954 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e99c2108-02b0-4bfe-8e64-d0f277c53eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:25.737665 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c2108-02b0-4bfe-8e64-d0f277c53eb1 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:25.744266 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c5792-0c43-486f-ad65-301d64e10160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:25.745203 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c5792-0c43-486f-ad65-301d64e10160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:25.745501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c5792-0c43-486f-ad65-301d64e10160 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:25.746298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c5792-0c43-486f-ad65-301d64e10160 admin admin] PATH_INFO: `/v3/projects/e6342a74f30a4a63a611697289ebf5b2` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:25.748175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c5792-0c43-486f-ad65-301d64e10160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6342a74f30a4a63a611697289ebf5b2)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:25.750765 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c5792-0c43-486f-ad65-301d64e10160 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:25.846139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:25.848865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-0b0c5792-0c43-486f-ad65-301d64e10160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6342a74f30a4a63a611697289ebf5b2', 'request_id': 'req-0b0c5792-0c43-486f-ad65-301d64e10160'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:25.858563 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b0ac0-06a0-4f90-a965-e1f1b0609fed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:25.863589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b0ac0-06a0-4f90-a965-e1f1b0609fed service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:25.866112 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-0b0c5792-0c43-486f-ad65-301d64e10160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6342a74f30a4a63a611697289ebf5b2', 'request_id': 'req-0b0c5792-0c43-486f-ad65-301d64e10160'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:25.883849 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-0b0c5792-0c43-486f-ad65-301d64e10160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6342a74f30a4a63a611697289ebf5b2', 'request_id': 'req-0b0c5792-0c43-486f-ad65-301d64e10160'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:25.888499 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1084/2153] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:25 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:46:25.889326 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b0ac0-06a0-4f90-a965-e1f1b0609fed service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:25.906172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b0ac0-06a0-4f90-a965-e1f1b0609fed service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:25.908017 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-0b0c5792-0c43-486f-ad65-301d64e10160 admin admin] The token cache is being invalidate because project e6342a74f30a4a63a611697289ebf5b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:25.908622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-0b0c5792-0c43-486f-ad65-301d64e10160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:25.910032 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-0b0c5792-0c43-486f-ad65-301d64e10160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:25.913850 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1070/2154] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:25 2022] DELETE /identity/v3/projects/e6342a74f30a4a63a611697289ebf5b2 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:25.920184 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c08e9-9718-4aff-873e-3a4e8a6e21f4 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:25.924027 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c08e9-9718-4aff-873e-3a4e8a6e21f4 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:25.924666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c08e9-9718-4aff-873e-3a4e8a6e21f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:26.004887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-5a4c08e9-9718-4aff-873e-3a4e8a6e21f4 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:26.008425 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-5a4c08e9-9718-4aff-873e-3a4e8a6e21f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:26.060717 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944fe78-629b-4d7b-89f9-a531b7e64769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:26.061429 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944fe78-629b-4d7b-89f9-a531b7e64769 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:26.061720 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944fe78-629b-4d7b-89f9-a531b7e64769 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:26.062004 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944fe78-629b-4d7b-89f9-a531b7e64769 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:26.063424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944fe78-629b-4d7b-89f9-a531b7e64769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:26.065627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944fe78-629b-4d7b-89f9-a531b7e64769 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:26.078275 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1085/2155] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:26.082240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944fe78-629b-4d7b-89f9-a531b7e64769 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:26.088538 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b944fe78-629b-4d7b-89f9-a531b7e64769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:26.103511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944fe78-629b-4d7b-89f9-a531b7e64769 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:26.109023 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8a541-37f1-4064-b295-ea58f630247a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:26.111885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8a541-37f1-4064-b295-ea58f630247a admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:26.112308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8a541-37f1-4064-b295-ea58f630247a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:26.112813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8a541-37f1-4064-b295-ea58f630247a admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:26.114073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8a541-37f1-4064-b295-ea58f630247a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:26.116941 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8a541-37f1-4064-b295-ea58f630247a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:26.134247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1086/2156] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:46:26.134973 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8a541-37f1-4064-b295-ea58f630247a admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:26.140789 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-81f8a541-37f1-4064-b295-ea58f630247a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:26.155607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1071/2157] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:26.158090 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c08e9-9718-4aff-873e-3a4e8a6e21f4 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:26.170993 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c08e9-9718-4aff-873e-3a4e8a6e21f4 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:26.176990 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7dec6-1ce7-4555-8920-9158af9c7d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:26.179633 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7dec6-1ce7-4555-8920-9158af9c7d80 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:26.179633 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7dec6-1ce7-4555-8920-9158af9c7d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:26.179633 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7dec6-1ce7-4555-8920-9158af9c7d80 admin admin] PATH_INFO: `/v3/projects/81d61acad188428dade1ddc886ce57ee/users/b6dee91fadde4caf950245a63f8b0cb2/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:26.189701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:26.196809 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7dec6-1ce7-4555-8920-9158af9c7d80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81d61acad188428dade1ddc886ce57ee, user_id=b6dee91fadde4caf950245a63f8b0cb2, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:26.199554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7dec6-1ce7-4555-8920-9158af9c7d80 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:26.219929 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1072/2158] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:26 2022] PUT /identity/v3/projects/81d61acad188428dade1ddc886ce57ee/users/b6dee91fadde4caf950245a63f8b0cb2/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:26.219929 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf86ec6-bb7b-45a7-9f90-387c5a41e202 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:26.219929 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf86ec6-bb7b-45a7-9f90-387c5a41e202 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:26.219929 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf86ec6-bb7b-45a7-9f90-387c5a41e202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:26.239771 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8a541-37f1-4064-b295-ea58f630247a admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:26.241294 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-3cf86ec6-bb7b-45a7-9f90-387c5a41e202 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:26.242513 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-3cf86ec6-bb7b-45a7-9f90-387c5a41e202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:26.301911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818052f2-e5ca-43ec-8858-e8d86a52b4a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:26.302582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818052f2-e5ca-43ec-8858-e8d86a52b4a8 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:26.302888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818052f2-e5ca-43ec-8858-e8d86a52b4a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:26.303166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818052f2-e5ca-43ec-8858-e8d86a52b4a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:26.305121 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-818052f2-e5ca-43ec-8858-e8d86a52b4a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:26.333761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1073/2159] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:26.333761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ed9dc-ac39-40cb-8de0-35ceba584578 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:26.333761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ed9dc-ac39-40cb-8de0-35ceba584578 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:26.333761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ed9dc-ac39-40cb-8de0-35ceba584578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:26.366975 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-f90ed9dc-ac39-40cb-8de0-35ceba584578 None None] MFA Rules not processed for user `b6dee91fadde4caf950245a63f8b0cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:26.368264 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f90ed9dc-ac39-40cb-8de0-35ceba584578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:26.397887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818052f2-e5ca-43ec-8858-e8d86a52b4a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:26.399428 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818052f2-e5ca-43ec-8858-e8d86a52b4a8 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:26.412308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1087/2160] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:26.412913 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818052f2-e5ca-43ec-8858-e8d86a52b4a8 service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:26.430114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818052f2-e5ca-43ec-8858-e8d86a52b4a8 service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:26.436000 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febbe4c-f5bd-492a-a109-91950b8795bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:26.436680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febbe4c-f5bd-492a-a109-91950b8795bc admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:26.436969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febbe4c-f5bd-492a-a109-91950b8795bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:26.437252 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febbe4c-f5bd-492a-a109-91950b8795bc admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:26.438679 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febbe4c-f5bd-492a-a109-91950b8795bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:46:26.438996 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febbe4c-f5bd-492a-a109-91950b8795bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:26.440646 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:26.441576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febbe4c-f5bd-492a-a109-91950b8795bc admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:26.464094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1088/2161] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:46:26.470109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febbe4c-f5bd-492a-a109-91950b8795bc admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:26.475001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1074/2162] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:26.475831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ed9dc-ac39-40cb-8de0-35ceba584578 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:26.489643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febbe4c-f5bd-492a-a109-91950b8795bc admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:26.494828 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ed9dc-ac39-40cb-8de0-35ceba584578 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:26.503700 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a10dcf-e9ce-40d3-9b24-af0853485032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:26.504529 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a10dcf-e9ce-40d3-9b24-af0853485032 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:26.504784 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a10dcf-e9ce-40d3-9b24-af0853485032 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:26.505055 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a10dcf-e9ce-40d3-9b24-af0853485032 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:26.505904 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a10dcf-e9ce-40d3-9b24-af0853485032 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:26.506463 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69b0e2-9916-4f9e-9d92-30bdcfd0014b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:26.507340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b0e2-9916-4f9e-9d92-30bdcfd0014b admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:26.507678 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b0e2-9916-4f9e-9d92-30bdcfd0014b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:26.507966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a10dcf-e9ce-40d3-9b24-af0853485032 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:26.508132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b0e2-9916-4f9e-9d92-30bdcfd0014b admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:26.508652 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69b0e2-9916-4f9e-9d92-30bdcfd0014b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:26.511535 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69b0e2-9916-4f9e-9d92-30bdcfd0014b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:26.551920 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1089/2163] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:26 2022] POST /identity/v3/projects => generated 387 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:46:26.552926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a10dcf-e9ce-40d3-9b24-af0853485032 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:26.565753 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1075/2164] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:26 2022] POST /identity/v3/projects => generated 377 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:26.568129 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69b0e2-9916-4f9e-9d92-30bdcfd0014b admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:26.641020 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a10dcf-e9ce-40d3-9b24-af0853485032 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:26.652863 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:26.659083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69b0e2-9916-4f9e-9d92-30bdcfd0014b admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:26.664850 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b60eb-72ab-446a-a456-e1bf6a4d6942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:26.666319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b60eb-72ab-446a-a456-e1bf6a4d6942 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:26.666319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b60eb-72ab-446a-a456-e1bf6a4d6942 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:26.666319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b60eb-72ab-446a-a456-e1bf6a4d6942 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:26.666994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b60eb-72ab-446a-a456-e1bf6a4d6942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:26.669033 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b60eb-72ab-446a-a456-e1bf6a4d6942 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:26.705112 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1076/2165] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:26 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:26.709443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b60eb-72ab-446a-a456-e1bf6a4d6942 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:26.721341 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b60eb-72ab-446a-a456-e1bf6a4d6942 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:26.726834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56571ce-46f4-4fec-ba51-41e24f545139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:26.727690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56571ce-46f4-4fec-ba51-41e24f545139 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:26.728191 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56571ce-46f4-4fec-ba51-41e24f545139 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:26.728532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56571ce-46f4-4fec-ba51-41e24f545139 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:26.729450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56571ce-46f4-4fec-ba51-41e24f545139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:26.731307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56571ce-46f4-4fec-ba51-41e24f545139 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:26.731765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eccdd0-1290-41c4-a3c8-09d15854d8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:26.732889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eccdd0-1290-41c4-a3c8-09d15854d8c8 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:26.733240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eccdd0-1290-41c4-a3c8-09d15854d8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:26.733811 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eccdd0-1290-41c4-a3c8-09d15854d8c8 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:26.736160 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eccdd0-1290-41c4-a3c8-09d15854d8c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:26.738862 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eccdd0-1290-41c4-a3c8-09d15854d8c8 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:26.749405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1077/2166] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:46:26.755243 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56571ce-46f4-4fec-ba51-41e24f545139 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:26.768188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1090/2167] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:26 2022] POST /identity/v3/users => generated 294 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:26.771983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56571ce-46f4-4fec-ba51-41e24f545139 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:26.772248 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eccdd0-1290-41c4-a3c8-09d15854d8c8 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:26.779159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b67b12-75f5-4f65-a913-a72f707b41a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:26.779879 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b67b12-75f5-4f65-a913-a72f707b41a0 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:26.780168 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b67b12-75f5-4f65-a913-a72f707b41a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:26.780454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b67b12-75f5-4f65-a913-a72f707b41a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:26.781285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b67b12-75f5-4f65-a913-a72f707b41a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:26.783053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b67b12-75f5-4f65-a913-a72f707b41a0 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:26.785955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eccdd0-1290-41c4-a3c8-09d15854d8c8 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:26.792776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4de5ad-c11c-4f56-a375-a04f1606b9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:26.793664 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4de5ad-c11c-4f56-a375-a04f1606b9bb admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:26.794048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4de5ad-c11c-4f56-a375-a04f1606b9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:26.794389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4de5ad-c11c-4f56-a375-a04f1606b9bb admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:26.795216 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4de5ad-c11c-4f56-a375-a04f1606b9bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:26.800385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1078/2168] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:46:26.800640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4de5ad-c11c-4f56-a375-a04f1606b9bb admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:26.801310 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b67b12-75f5-4f65-a913-a72f707b41a0 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:26.814386 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b67b12-75f5-4f65-a913-a72f707b41a0 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:26.817970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1091/2169] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:26.821991 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4de5ad-c11c-4f56-a375-a04f1606b9bb admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:26.825759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f13c3-c35a-42c1-ace5-eea94339539a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:26.827979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f13c3-c35a-42c1-ace5-eea94339539a admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:26.828062 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f13c3-c35a-42c1-ace5-eea94339539a admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:26.828938 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f13c3-c35a-42c1-ace5-eea94339539a admin admin] PATH_INFO: `/v3/projects/bee61c6de6b04cca9b0d984702e337cd/users/e6b23ddcff6e45359b46007698d402f6/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:26.845302 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4de5ad-c11c-4f56-a375-a04f1606b9bb admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:26.852879 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f13c3-c35a-42c1-ace5-eea94339539a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bee61c6de6b04cca9b0d984702e337cd, user_id=e6b23ddcff6e45359b46007698d402f6, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:26.855897 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f13c3-c35a-42c1-ace5-eea94339539a admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:26.858690 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c460542-6c45-46b0-aba8-c1f554f6b56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:26.860498 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c460542-6c45-46b0-aba8-c1f554f6b56e admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:26.861173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c460542-6c45-46b0-aba8-c1f554f6b56e admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:26.861745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c460542-6c45-46b0-aba8-c1f554f6b56e admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:26.864032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c460542-6c45-46b0-aba8-c1f554f6b56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:26.867839 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c460542-6c45-46b0-aba8-c1f554f6b56e admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:26.885588 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1079/2170] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:26 2022] PUT /identity/v3/projects/bee61c6de6b04cca9b0d984702e337cd/users/e6b23ddcff6e45359b46007698d402f6/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:26.886885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1092/2171] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:46:26.890014 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303912fc-6437-4dbe-9f98-e1ec69366b73 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:26.890450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303912fc-6437-4dbe-9f98-e1ec69366b73 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:26.890854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303912fc-6437-4dbe-9f98-e1ec69366b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:26.891626 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c460542-6c45-46b0-aba8-c1f554f6b56e admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:26.903977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c460542-6c45-46b0-aba8-c1f554f6b56e admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:26.940303 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-303912fc-6437-4dbe-9f98-e1ec69366b73 None None] MFA Rules not processed for user `e6b23ddcff6e45359b46007698d402f6`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:26.943533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-303912fc-6437-4dbe-9f98-e1ec69366b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:27.004752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde4ad3-2f8d-471b-9e05-a078889326d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:27.005744 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde4ad3-2f8d-471b-9e05-a078889326d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:27.006104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde4ad3-2f8d-471b-9e05-a078889326d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:27.006389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde4ad3-2f8d-471b-9e05-a078889326d1 admin admin] PATH_INFO: `/v3/projects/97fd758460354b82a5f917ef8cc9951b/users/3c745414680b4594a1a4b693d7f31545/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:27.016360 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:27.029642 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde4ad3-2f8d-471b-9e05-a078889326d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97fd758460354b82a5f917ef8cc9951b, user_id=3c745414680b4594a1a4b693d7f31545, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:27.033222 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde4ad3-2f8d-471b-9e05-a078889326d1 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:27.048929 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1080/2172] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:27.051340 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1093/2173] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:26 2022] PUT /identity/v3/projects/97fd758460354b82a5f917ef8cc9951b/users/3c745414680b4594a1a4b693d7f31545/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:27.052418 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303912fc-6437-4dbe-9f98-e1ec69366b73 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:27.062308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035f65a-f1a3-4ab2-9e0d-9240da9eb9a2 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:27.062725 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035f65a-f1a3-4ab2-9e0d-9240da9eb9a2 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:27.063961 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035f65a-f1a3-4ab2-9e0d-9240da9eb9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:27.070146 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303912fc-6437-4dbe-9f98-e1ec69366b73 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:27.101838 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d035f65a-f1a3-4ab2-9e0d-9240da9eb9a2 None None] MFA Rules not processed for user `3c745414680b4594a1a4b693d7f31545`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:27.103129 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d035f65a-f1a3-4ab2-9e0d-9240da9eb9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:27.169694 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3b363-9f0e-4726-b6c7-43e5f5be2f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:27.170005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3b363-9f0e-4726-b6c7-43e5f5be2f04 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:27.170299 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3b363-9f0e-4726-b6c7-43e5f5be2f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:27.170610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3b363-9f0e-4726-b6c7-43e5f5be2f04 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:27.171293 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3b363-9f0e-4726-b6c7-43e5f5be2f04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:27.173545 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3b363-9f0e-4726-b6c7-43e5f5be2f04 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:27.181663 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:27.204995 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1081/2174] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:27 2022] POST /identity/v3/projects => generated 377 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:27.211193 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3b363-9f0e-4726-b6c7-43e5f5be2f04 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:27.213579 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1094/2175] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:27.218085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035f65a-f1a3-4ab2-9e0d-9240da9eb9a2 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:27.234363 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035f65a-f1a3-4ab2-9e0d-9240da9eb9a2 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:27.314253 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3b363-9f0e-4726-b6c7-43e5f5be2f04 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:27.321864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f1a28-0cca-4d9f-9562-47d599286f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:27.323171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f1a28-0cca-4d9f-9562-47d599286f0b admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:27.323689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f1a28-0cca-4d9f-9562-47d599286f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:27.324198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f1a28-0cca-4d9f-9562-47d599286f0b admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:27.325527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f1a28-0cca-4d9f-9562-47d599286f0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:27.329382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f1a28-0cca-4d9f-9562-47d599286f0b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:27.333347 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa133c0-8fa5-4d38-9525-e2f09c0b280f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:27.334359 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa133c0-8fa5-4d38-9525-e2f09c0b280f admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:27.334821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa133c0-8fa5-4d38-9525-e2f09c0b280f admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:27.335752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa133c0-8fa5-4d38-9525-e2f09c0b280f admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:27.337034 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa133c0-8fa5-4d38-9525-e2f09c0b280f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:27.340895 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa133c0-8fa5-4d38-9525-e2f09c0b280f admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:27.368339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1082/2176] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:27 2022] POST /identity/v3/users => generated 295 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:46:27.371813 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f1a28-0cca-4d9f-9562-47d599286f0b admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:27.388425 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1095/2177] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:27 2022] POST /identity/v3/projects => generated 385 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:46:27.390581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f1a28-0cca-4d9f-9562-47d599286f0b admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:27.394371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa133c0-8fa5-4d38-9525-e2f09c0b280f admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:27.505872 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa133c0-8fa5-4d38-9525-e2f09c0b280f admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:27.508310 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c4a93-07ef-42ab-9d5b-175f3f6efac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:27.509104 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c4a93-07ef-42ab-9d5b-175f3f6efac7 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:27.509551 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c4a93-07ef-42ab-9d5b-175f3f6efac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:27.510017 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c4a93-07ef-42ab-9d5b-175f3f6efac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:27.510943 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c4a93-07ef-42ab-9d5b-175f3f6efac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:27.511645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae30a3-b0ef-44fa-9a99-70b02f15c54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:27.512341 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae30a3-b0ef-44fa-9a99-70b02f15c54b admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:27.512604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae30a3-b0ef-44fa-9a99-70b02f15c54b admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:27.512881 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae30a3-b0ef-44fa-9a99-70b02f15c54b admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:27.513523 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c4a93-07ef-42ab-9d5b-175f3f6efac7 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:27.513708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae30a3-b0ef-44fa-9a99-70b02f15c54b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:27.515621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ae30a3-b0ef-44fa-9a99-70b02f15c54b admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:27.526655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1083/2178] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:27.530981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c4a93-07ef-42ab-9d5b-175f3f6efac7 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:27.538832 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:27.543260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1096/2179] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:27 2022] POST /identity/v3/users => generated 299 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:27.544375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae30a3-b0ef-44fa-9a99-70b02f15c54b admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:27.544623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c4a93-07ef-42ab-9d5b-175f3f6efac7 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:27.553173 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7dbeb-5755-41ab-8d29-1bb75b5176eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:27.554099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7dbeb-5755-41ab-8d29-1bb75b5176eb admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:27.554380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7dbeb-5755-41ab-8d29-1bb75b5176eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:27.554658 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7dbeb-5755-41ab-8d29-1bb75b5176eb admin admin] PATH_INFO: `/v3/projects/bd98346cd7634c1195be359c27172368/users/34eb330cbb364212afe626f8a62dd2f4/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:27.561706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ae30a3-b0ef-44fa-9a99-70b02f15c54b admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:27.569908 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea49bf0-6e21-49d6-a7c2-1f8dced83785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:27.570602 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea49bf0-6e21-49d6-a7c2-1f8dced83785 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:27.570877 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea49bf0-6e21-49d6-a7c2-1f8dced83785 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:27.571157 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea49bf0-6e21-49d6-a7c2-1f8dced83785 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:27.571939 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea49bf0-6e21-49d6-a7c2-1f8dced83785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:27.573781 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea49bf0-6e21-49d6-a7c2-1f8dced83785 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:27.577898 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7dbeb-5755-41ab-8d29-1bb75b5176eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd98346cd7634c1195be359c27172368, user_id=34eb330cbb364212afe626f8a62dd2f4, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:27.585261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1097/2180] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:27.590280 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7dbeb-5755-41ab-8d29-1bb75b5176eb admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:27.591338 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea49bf0-6e21-49d6-a7c2-1f8dced83785 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:27.602607 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea49bf0-6e21-49d6-a7c2-1f8dced83785 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:27.620036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1084/2181] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:27 2022] PUT /identity/v3/projects/bd98346cd7634c1195be359c27172368/users/34eb330cbb364212afe626f8a62dd2f4/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:27.626762 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7dbeb-5755-41ab-8d29-1bb75b5176eb admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:27.685637 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:27.702292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0459f-fe96-432d-8999-0f96a25d7d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:27.703194 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0459f-fe96-432d-8999-0f96a25d7d50 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:27.703539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0459f-fe96-432d-8999-0f96a25d7d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:27.704188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0459f-fe96-432d-8999-0f96a25d7d50 admin admin] PATH_INFO: `/v3/projects/b576e7fc24754f10b3684087e9d5c8b6/users/7f6c7efa12c04128aff76ea56a446edd/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:27.727226 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0459f-fe96-432d-8999-0f96a25d7d50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b576e7fc24754f10b3684087e9d5c8b6, user_id=7f6c7efa12c04128aff76ea56a446edd, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:27.729161 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7dbeb-5755-41ab-8d29-1bb75b5176eb admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:27.730321 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0459f-fe96-432d-8999-0f96a25d7d50 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:27.737140 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42756c7-a817-45fa-9908-2e9cc33a7aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:27.737956 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42756c7-a817-45fa-9908-2e9cc33a7aae admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:27.738283 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42756c7-a817-45fa-9908-2e9cc33a7aae admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:27.738532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42756c7-a817-45fa-9908-2e9cc33a7aae admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:27.739919 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42756c7-a817-45fa-9908-2e9cc33a7aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:27.742030 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42756c7-a817-45fa-9908-2e9cc33a7aae admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:27.751962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1098/2182] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:27 2022] PUT /identity/v3/projects/b576e7fc24754f10b3684087e9d5c8b6/users/7f6c7efa12c04128aff76ea56a446edd/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:27.759400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1085/2183] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:27.759633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0459f-fe96-432d-8999-0f96a25d7d50 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:27.763916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42756c7-a817-45fa-9908-2e9cc33a7aae admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:27.776265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42756c7-a817-45fa-9908-2e9cc33a7aae admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:27.859224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0459f-fe96-432d-8999-0f96a25d7d50 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:27.864891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7622af-3f35-49ac-9405-6c5d8e53a53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:27.865616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7622af-3f35-49ac-9405-6c5d8e53a53f admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:27.865881 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7622af-3f35-49ac-9405-6c5d8e53a53f admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:27.866162 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7622af-3f35-49ac-9405-6c5d8e53a53f admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:27.866925 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7622af-3f35-49ac-9405-6c5d8e53a53f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:27.867067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dad268-9e79-4ac1-a521-37cacc0e3c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:27.867972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dad268-9e79-4ac1-a521-37cacc0e3c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:27.868257 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dad268-9e79-4ac1-a521-37cacc0e3c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:27.868535 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dad268-9e79-4ac1-a521-37cacc0e3c12 admin admin] PATH_INFO: `/v3/projects/bd98346cd7634c1195be359c27172368/users/34eb330cbb364212afe626f8a62dd2f4/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:27.868640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7622af-3f35-49ac-9405-6c5d8e53a53f admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:27.870924 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dad268-9e79-4ac1-a521-37cacc0e3c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd98346cd7634c1195be359c27172368, user_id=34eb330cbb364212afe626f8a62dd2f4, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:27.873946 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dad268-9e79-4ac1-a521-37cacc0e3c12 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:27.884389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1099/2184] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:27.888707 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7622af-3f35-49ac-9405-6c5d8e53a53f admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:27.892281 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1086/2185] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:27 2022] PUT /identity/v3/projects/bd98346cd7634c1195be359c27172368/users/34eb330cbb364212afe626f8a62dd2f4/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:27.901697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7622af-3f35-49ac-9405-6c5d8e53a53f admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:27.905094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fdca6-39b6-4360-8c06-014b271bfd22 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:27.905398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fdca6-39b6-4360-8c06-014b271bfd22 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:27.906015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649fdca6-39b6-4360-8c06-014b271bfd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:27.951069 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-649fdca6-39b6-4360-8c06-014b271bfd22 None None] MFA Rules not processed for user `34eb330cbb364212afe626f8a62dd2f4`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:27.952351 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-649fdca6-39b6-4360-8c06-014b271bfd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:27.986940 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6c55f-548e-475b-af48-b72b01dcbb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:27.987907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6c55f-548e-475b-af48-b72b01dcbb50 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:27.988187 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6c55f-548e-475b-af48-b72b01dcbb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:27.988460 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6c55f-548e-475b-af48-b72b01dcbb50 admin admin] PATH_INFO: `/v3/projects/b576e7fc24754f10b3684087e9d5c8b6/users/7f6c7efa12c04128aff76ea56a446edd/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:27.990705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6c55f-548e-475b-af48-b72b01dcbb50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b576e7fc24754f10b3684087e9d5c8b6, user_id=7f6c7efa12c04128aff76ea56a446edd, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:27.993470 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6c55f-548e-475b-af48-b72b01dcbb50 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:28.008219 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1100/2186] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:27 2022] PUT /identity/v3/projects/b576e7fc24754f10b3684087e9d5c8b6/users/7f6c7efa12c04128aff76ea56a446edd/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:28.023553 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603631f-6d22-45d6-a482-ed90a6e94ece None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:28.024087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603631f-6d22-45d6-a482-ed90a6e94ece None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:28.024370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603631f-6d22-45d6-a482-ed90a6e94ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:28.029799 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:28.041444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1087/2187] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:27 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:28.055463 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fdca6-39b6-4360-8c06-014b271bfd22 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:28.056987 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-649fdca6-39b6-4360-8c06-014b271bfd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:28.071489 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-1603631f-6d22-45d6-a482-ed90a6e94ece None None] MFA Rules not processed for user `7f6c7efa12c04128aff76ea56a446edd`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:28.072958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-1603631f-6d22-45d6-a482-ed90a6e94ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:28.147609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649fdca6-39b6-4360-8c06-014b271bfd22 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:28.163275 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:28.183085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1101/2188] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:28.240183 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603631f-6d22-45d6-a482-ed90a6e94ece None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:28.245352 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183cc4d-4ffc-409d-a38d-86b199bd8d25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:28.246273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183cc4d-4ffc-409d-a38d-86b199bd8d25 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:28.246629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183cc4d-4ffc-409d-a38d-86b199bd8d25 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:28.247116 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183cc4d-4ffc-409d-a38d-86b199bd8d25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:28.261689 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603631f-6d22-45d6-a482-ed90a6e94ece None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:28.269317 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00df71-8caf-4ae9-acda-85e2affe353b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:28.270434 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00df71-8caf-4ae9-acda-85e2affe353b service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:28.274402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00df71-8caf-4ae9-acda-85e2affe353b service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:28.274695 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00df71-8caf-4ae9-acda-85e2affe353b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:28.288493 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df00df71-8caf-4ae9-acda-85e2affe353b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:28.289988 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df00df71-8caf-4ae9-acda-85e2affe353b service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:28.306958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1102/2189] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:28.343212 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00df71-8caf-4ae9-acda-85e2affe353b service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:28.359617 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2183cc4d-4ffc-409d-a38d-86b199bd8d25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:28.363095 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2183cc4d-4ffc-409d-a38d-86b199bd8d25 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:28.378163 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1088/2190] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:46:28.514190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00df71-8caf-4ae9-acda-85e2affe353b service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:28.523705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1e36b-1568-47e2-bcf8-759db888f860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:28.524592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1e36b-1568-47e2-bcf8-759db888f860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:28.524941 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1e36b-1568-47e2-bcf8-759db888f860 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:28.525223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1e36b-1568-47e2-bcf8-759db888f860 admin admin] PATH_INFO: `/v3/users/3c745414680b4594a1a4b693d7f31545` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:28.526566 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1e36b-1568-47e2-bcf8-759db888f860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c745414680b4594a1a4b693d7f31545)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:28.529517 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1e36b-1568-47e2-bcf8-759db888f860 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:28.635801 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c7f1e36b-1568-47e2-bcf8-759db888f860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c745414680b4594a1a4b693d7f31545'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:28.661953 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c7f1e36b-1568-47e2-bcf8-759db888f860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c745414680b4594a1a4b693d7f31545'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:28.673124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c7f1e36b-1568-47e2-bcf8-759db888f860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c745414680b4594a1a4b693d7f31545'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:28.686759 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c7f1e36b-1568-47e2-bcf8-759db888f860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c745414680b4594a1a4b693d7f31545'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:28.690381 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c7f1e36b-1568-47e2-bcf8-759db888f860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c745414680b4594a1a4b693d7f31545'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:28.703921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1103/2191] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:28 2022] DELETE /identity/v3/users/3c745414680b4594a1a4b693d7f31545 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:28.709059 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188c2cc-3533-4e72-9e2c-716c16176916 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:28.709415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188c2cc-3533-4e72-9e2c-716c16176916 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:28.709690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188c2cc-3533-4e72-9e2c-716c16176916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:28.733146 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-e188c2cc-3533-4e72-9e2c-716c16176916 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:28.734558 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e188c2cc-3533-4e72-9e2c-716c16176916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:28.821842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1089/2192] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:28.838470 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1e36b-1568-47e2-bcf8-759db888f860 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:28.841090 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-c7f1e36b-1568-47e2-bcf8-759db888f860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:28.971124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1e36b-1568-47e2-bcf8-759db888f860 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:28.985291 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59ecdd-ebbf-4e4d-8e3e-f10d96d6f8cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:28.985996 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59ecdd-ebbf-4e4d-8e3e-f10d96d6f8cc service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:28.987463 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59ecdd-ebbf-4e4d-8e3e-f10d96d6f8cc service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:28.988074 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59ecdd-ebbf-4e4d-8e3e-f10d96d6f8cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:28.999794 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59ecdd-ebbf-4e4d-8e3e-f10d96d6f8cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:29.001282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59ecdd-ebbf-4e4d-8e3e-f10d96d6f8cc service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:29.021105 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1104/2193] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:29.565224 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188c2cc-3533-4e72-9e2c-716c16176916 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:29.567589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e188c2cc-3533-4e72-9e2c-716c16176916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:29.581695 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188c2cc-3533-4e72-9e2c-716c16176916 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:29.589196 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86529146-6f51-487b-81fa-4a3ccc4f9857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:29.590180 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86529146-6f51-487b-81fa-4a3ccc4f9857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:29.590462 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86529146-6f51-487b-81fa-4a3ccc4f9857 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:29.590789 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86529146-6f51-487b-81fa-4a3ccc4f9857 admin admin] PATH_INFO: `/v3/projects/97fd758460354b82a5f917ef8cc9951b` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:29.592123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86529146-6f51-487b-81fa-4a3ccc4f9857 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97fd758460354b82a5f917ef8cc9951b)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:29.594106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86529146-6f51-487b-81fa-4a3ccc4f9857 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:29.595926 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:29.655933 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-86529146-6f51-487b-81fa-4a3ccc4f9857 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97fd758460354b82a5f917ef8cc9951b', 'request_id': 'req-86529146-6f51-487b-81fa-4a3ccc4f9857'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:29.670667 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-86529146-6f51-487b-81fa-4a3ccc4f9857 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97fd758460354b82a5f917ef8cc9951b', 'request_id': 'req-86529146-6f51-487b-81fa-4a3ccc4f9857'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:29.681546 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-86529146-6f51-487b-81fa-4a3ccc4f9857 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97fd758460354b82a5f917ef8cc9951b', 'request_id': 'req-86529146-6f51-487b-81fa-4a3ccc4f9857'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:29.696953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-86529146-6f51-487b-81fa-4a3ccc4f9857 admin admin] The token cache is being invalidate because project 97fd758460354b82a5f917ef8cc9951b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:29.697424 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-86529146-6f51-487b-81fa-4a3ccc4f9857 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:29.698456 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-86529146-6f51-487b-81fa-4a3ccc4f9857 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:29.701213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1090/2194] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:29 2022] DELETE /identity/v3/projects/97fd758460354b82a5f917ef8cc9951b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:29.705389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59ecdd-ebbf-4e4d-8e3e-f10d96d6f8cc service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:29.710422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:29.732029 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59ecdd-ebbf-4e4d-8e3e-f10d96d6f8cc service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:29.737232 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86529146-6f51-487b-81fa-4a3ccc4f9857 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:29.738832 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-86529146-6f51-487b-81fa-4a3ccc4f9857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:29.817347 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42c21d-ee55-47d7-b995-3fe4853d17df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:29.818688 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42c21d-ee55-47d7-b995-3fe4853d17df admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:29.819195 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42c21d-ee55-47d7-b995-3fe4853d17df admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:29.819735 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42c21d-ee55-47d7-b995-3fe4853d17df admin admin] PATH_INFO: `/v3/users/7f6c7efa12c04128aff76ea56a446edd` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:29.821764 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42c21d-ee55-47d7-b995-3fe4853d17df admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f6c7efa12c04128aff76ea56a446edd)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:29.825474 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42c21d-ee55-47d7-b995-3fe4853d17df admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:29.828764 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86529146-6f51-487b-81fa-4a3ccc4f9857 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:29.833587 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c6a6f-ffee-4c68-a4c1-c53fb7b4e93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:29.834463 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c6a6f-ffee-4c68-a4c1-c53fb7b4e93b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:29.834743 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c6a6f-ffee-4c68-a4c1-c53fb7b4e93b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:29.835019 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c6a6f-ffee-4c68-a4c1-c53fb7b4e93b admin admin] PATH_INFO: `/v3/users/9d221d89ac944f21a84331c084b05ced` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:29.836299 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c6a6f-ffee-4c68-a4c1-c53fb7b4e93b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d221d89ac944f21a84331c084b05ced)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:29.839371 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685c6a6f-ffee-4c68-a4c1-c53fb7b4e93b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:29.892619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-2f42c21d-ee55-47d7-b995-3fe4853d17df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f6c7efa12c04128aff76ea56a446edd'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:29.904248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-685c6a6f-ffee-4c68-a4c1-c53fb7b4e93b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d221d89ac944f21a84331c084b05ced'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:29.908903 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-2f42c21d-ee55-47d7-b995-3fe4853d17df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f6c7efa12c04128aff76ea56a446edd'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:29.919016 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-2f42c21d-ee55-47d7-b995-3fe4853d17df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f6c7efa12c04128aff76ea56a446edd'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:29.930240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-2f42c21d-ee55-47d7-b995-3fe4853d17df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f6c7efa12c04128aff76ea56a446edd'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:29.931078 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-2f42c21d-ee55-47d7-b995-3fe4853d17df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f6c7efa12c04128aff76ea56a446edd'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:29.935768 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-685c6a6f-ffee-4c68-a4c1-c53fb7b4e93b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d221d89ac944f21a84331c084b05ced'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:29.944350 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1105/2195] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:29 2022] DELETE /identity/v3/users/7f6c7efa12c04128aff76ea56a446edd => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:29.951613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-685c6a6f-ffee-4c68-a4c1-c53fb7b4e93b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d221d89ac944f21a84331c084b05ced'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:29.952760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-685c6a6f-ffee-4c68-a4c1-c53fb7b4e93b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d221d89ac944f21a84331c084b05ced'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:29.965272 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-685c6a6f-ffee-4c68-a4c1-c53fb7b4e93b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d221d89ac944f21a84331c084b05ced'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:29.972378 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1091/2196] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:29 2022] DELETE /identity/v3/users/9d221d89ac944f21a84331c084b05ced => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:29.977608 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b554c-7990-4e18-8d77-419e5ade6d25 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:29.978170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b554c-7990-4e18-8d77-419e5ade6d25 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:29.978561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b554c-7990-4e18-8d77-419e5ade6d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:30.009845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-8e6b554c-7990-4e18-8d77-419e5ade6d25 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:30.012057 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-8e6b554c-7990-4e18-8d77-419e5ade6d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:30.111120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1106/2197] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:30.125935 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c6a6f-ffee-4c68-a4c1-c53fb7b4e93b admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:30.127489 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-685c6a6f-ffee-4c68-a4c1-c53fb7b4e93b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:30.139029 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:30.211470 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c6a6f-ffee-4c68-a4c1-c53fb7b4e93b admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:30.217113 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe17c60-45e1-42f5-b064-d1200240ad6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:30.218147 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe17c60-45e1-42f5-b064-d1200240ad6d service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:30.218586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe17c60-45e1-42f5-b064-d1200240ad6d service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:30.219075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe17c60-45e1-42f5-b064-d1200240ad6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:30.235287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe17c60-45e1-42f5-b064-d1200240ad6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:30.236775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe17c60-45e1-42f5-b064-d1200240ad6d service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:30.248041 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1092/2198] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:46:30.316175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b554c-7990-4e18-8d77-419e5ade6d25 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:30.319754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-8e6b554c-7990-4e18-8d77-419e5ade6d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:30.346328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b554c-7990-4e18-8d77-419e5ade6d25 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:30.358492 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3e23d-a6a7-4b53-88c6-4a37b34db220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:30.360194 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3e23d-a6a7-4b53-88c6-4a37b34db220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:30.360630 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3e23d-a6a7-4b53-88c6-4a37b34db220 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:30.362248 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3e23d-a6a7-4b53-88c6-4a37b34db220 admin admin] PATH_INFO: `/v3/projects/b576e7fc24754f10b3684087e9d5c8b6` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:30.363137 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3e23d-a6a7-4b53-88c6-4a37b34db220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b576e7fc24754f10b3684087e9d5c8b6)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:30.369440 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3e23d-a6a7-4b53-88c6-4a37b34db220 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:30.396862 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:30.456702 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d6f3e23d-a6a7-4b53-88c6-4a37b34db220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b576e7fc24754f10b3684087e9d5c8b6', 'request_id': 'req-d6f3e23d-a6a7-4b53-88c6-4a37b34db220'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:30.468993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d6f3e23d-a6a7-4b53-88c6-4a37b34db220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b576e7fc24754f10b3684087e9d5c8b6', 'request_id': 'req-d6f3e23d-a6a7-4b53-88c6-4a37b34db220'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:30.483307 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d6f3e23d-a6a7-4b53-88c6-4a37b34db220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b576e7fc24754f10b3684087e9d5c8b6', 'request_id': 'req-d6f3e23d-a6a7-4b53-88c6-4a37b34db220'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:30.510263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d6f3e23d-a6a7-4b53-88c6-4a37b34db220 admin admin] The token cache is being invalidate because project b576e7fc24754f10b3684087e9d5c8b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:30.511257 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d6f3e23d-a6a7-4b53-88c6-4a37b34db220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:30.512569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d6f3e23d-a6a7-4b53-88c6-4a37b34db220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:30.522135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1107/2199] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:30 2022] DELETE /identity/v3/projects/b576e7fc24754f10b3684087e9d5c8b6 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:30.523394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0ffa5-1264-4c37-991a-05b001848d3b None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:30.523699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0ffa5-1264-4c37-991a-05b001848d3b None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:30.524148 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0ffa5-1264-4c37-991a-05b001848d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:30.558227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-aaf0ffa5-1264-4c37-991a-05b001848d3b None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:30.559630 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-aaf0ffa5-1264-4c37-991a-05b001848d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:30.661713 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1093/2200] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:30.670483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd2c8f-2ec8-4ea1-adb3-7dcacd04e285 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:30.670826 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd2c8f-2ec8-4ea1-adb3-7dcacd04e285 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:30.670975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd2c8f-2ec8-4ea1-adb3-7dcacd04e285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:30.698410 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-44cd2c8f-2ec8-4ea1-adb3-7dcacd04e285 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:30.698410 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-44cd2c8f-2ec8-4ea1-adb3-7dcacd04e285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:30.714145 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1108/2201] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:30.715066 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0ffa5-1264-4c37-991a-05b001848d3b None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:30.728893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0ffa5-1264-4c37-991a-05b001848d3b None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:30.738844 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea265928-b14f-49cf-899d-02a99f641399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:30.739957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea265928-b14f-49cf-899d-02a99f641399 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:30.740399 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea265928-b14f-49cf-899d-02a99f641399 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:30.741032 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea265928-b14f-49cf-899d-02a99f641399 admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:30.742327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea265928-b14f-49cf-899d-02a99f641399 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:46:30.742927 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea265928-b14f-49cf-899d-02a99f641399 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:30.744971 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea265928-b14f-49cf-899d-02a99f641399 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:30.786925 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd2c8f-2ec8-4ea1-adb3-7dcacd04e285 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:30.798222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1094/2202] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 11:46:30.817169 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cd2c8f-2ec8-4ea1-adb3-7dcacd04e285 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:30.827434 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2783ec-21a2-4ac3-9a67-113adeee8a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:30.828643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2783ec-21a2-4ac3-9a67-113adeee8a19 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:30.829114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2783ec-21a2-4ac3-9a67-113adeee8a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:30.829593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2783ec-21a2-4ac3-9a67-113adeee8a19 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:30.830509 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2783ec-21a2-4ac3-9a67-113adeee8a19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:30.832867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2783ec-21a2-4ac3-9a67-113adeee8a19 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:30.896096 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1109/2203] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:30 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:30.903489 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea265928-b14f-49cf-899d-02a99f641399 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:30.904380 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2783ec-21a2-4ac3-9a67-113adeee8a19 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:30.905532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ea265928-b14f-49cf-899d-02a99f641399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:30.921786 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea265928-b14f-49cf-899d-02a99f641399 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:31.025311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c81441-7f45-4166-9e3e-1e764f00d605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:31.026329 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c81441-7f45-4166-9e3e-1e764f00d605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:31.026818 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c81441-7f45-4166-9e3e-1e764f00d605 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:31.027178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c81441-7f45-4166-9e3e-1e764f00d605 admin admin] PATH_INFO: `/v3/projects/cb56dbfeb9434e039c9708b86882f4ec` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:31.028887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c81441-7f45-4166-9e3e-1e764f00d605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb56dbfeb9434e039c9708b86882f4ec)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:31.032397 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c81441-7f45-4166-9e3e-1e764f00d605 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:31.095006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2783ec-21a2-4ac3-9a67-113adeee8a19 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:31.122851 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:31.138542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-45c81441-7f45-4166-9e3e-1e764f00d605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb56dbfeb9434e039c9708b86882f4ec', 'request_id': 'req-45c81441-7f45-4166-9e3e-1e764f00d605'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:31.151456 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-45c81441-7f45-4166-9e3e-1e764f00d605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb56dbfeb9434e039c9708b86882f4ec', 'request_id': 'req-45c81441-7f45-4166-9e3e-1e764f00d605'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:31.164397 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-45c81441-7f45-4166-9e3e-1e764f00d605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb56dbfeb9434e039c9708b86882f4ec', 'request_id': 'req-45c81441-7f45-4166-9e3e-1e764f00d605'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:31.182769 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8ea94-6dda-4e77-9111-efee5c4da78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:31.183743 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8ea94-6dda-4e77-9111-efee5c4da78d admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:31.184202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8ea94-6dda-4e77-9111-efee5c4da78d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:31.184494 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8ea94-6dda-4e77-9111-efee5c4da78d admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:31.185485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8ea94-6dda-4e77-9111-efee5c4da78d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:31.188008 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8ea94-6dda-4e77-9111-efee5c4da78d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:31.188307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-45c81441-7f45-4166-9e3e-1e764f00d605 admin admin] The token cache is being invalidate because project cb56dbfeb9434e039c9708b86882f4ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:31.188883 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-45c81441-7f45-4166-9e3e-1e764f00d605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:31.189781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-45c81441-7f45-4166-9e3e-1e764f00d605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:31.192347 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1095/2204] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:30 2022] DELETE /identity/v3/projects/cb56dbfeb9434e039c9708b86882f4ec => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:31.197145 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c81441-7f45-4166-9e3e-1e764f00d605 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:31.199148 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-45c81441-7f45-4166-9e3e-1e764f00d605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:31.216739 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c81441-7f45-4166-9e3e-1e764f00d605 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:31.223824 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1110/2205] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:30 2022] POST /identity/v3/users => generated 286 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:31.225541 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8ea94-6dda-4e77-9111-efee5c4da78d admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:31.230150 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-64f8ea94-6dda-4e77-9111-efee5c4da78d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:31.232197 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03968ab-03bc-4d6d-9805-ccd62e024dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:31.233429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03968ab-03bc-4d6d-9805-ccd62e024dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:31.233859 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03968ab-03bc-4d6d-9805-ccd62e024dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:31.234280 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03968ab-03bc-4d6d-9805-ccd62e024dcb admin admin] PATH_INFO: `/v3/users/8e78427b641a415482dfbaabcd7812ab` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:31.236139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03968ab-03bc-4d6d-9805-ccd62e024dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e78427b641a415482dfbaabcd7812ab)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:31.238364 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03968ab-03bc-4d6d-9805-ccd62e024dcb admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:31.260741 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8ea94-6dda-4e77-9111-efee5c4da78d admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:31.270110 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f2d76-ca40-4a3f-9ad2-a1500dabb585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:31.271024 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f2d76-ca40-4a3f-9ad2-a1500dabb585 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:31.271430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f2d76-ca40-4a3f-9ad2-a1500dabb585 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:31.271783 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f2d76-ca40-4a3f-9ad2-a1500dabb585 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:31.273303 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f2d76-ca40-4a3f-9ad2-a1500dabb585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:31.278522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f2d76-ca40-4a3f-9ad2-a1500dabb585 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:31.306394 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1111/2206] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:46:31.311415 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f2d76-ca40-4a3f-9ad2-a1500dabb585 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:31.317499 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:31.323966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f2d76-ca40-4a3f-9ad2-a1500dabb585 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:31.334194 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9321c31d-0eb8-4493-a99e-a1bdea2a1ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:31.335188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321c31d-0eb8-4493-a99e-a1bdea2a1ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:31.335555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321c31d-0eb8-4493-a99e-a1bdea2a1ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:31.335888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321c31d-0eb8-4493-a99e-a1bdea2a1ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:31.336654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9321c31d-0eb8-4493-a99e-a1bdea2a1ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:31.337320 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-e03968ab-03bc-4d6d-9805-ccd62e024dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e78427b641a415482dfbaabcd7812ab'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:31.340594 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9321c31d-0eb8-4493-a99e-a1bdea2a1ba4 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:31.365122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1112/2207] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:31.366215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9321c31d-0eb8-4493-a99e-a1bdea2a1ba4 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:31.380216 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-e03968ab-03bc-4d6d-9805-ccd62e024dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e78427b641a415482dfbaabcd7812ab'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:31.403820 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9321c31d-0eb8-4493-a99e-a1bdea2a1ba4 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:31.410637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-e03968ab-03bc-4d6d-9805-ccd62e024dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e78427b641a415482dfbaabcd7812ab'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:31.411545 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-e03968ab-03bc-4d6d-9805-ccd62e024dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e78427b641a415482dfbaabcd7812ab'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:31.418648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c2c5d-6c51-4135-86d3-96092ecb922b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:31.419614 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c2c5d-6c51-4135-86d3-96092ecb922b admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:31.419917 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c2c5d-6c51-4135-86d3-96092ecb922b admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:31.420187 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c2c5d-6c51-4135-86d3-96092ecb922b admin admin] PATH_INFO: `/v3/projects/93d2fd56c8f142a38927ad957869c5d2/users/5b02b318056e4832aca3186795cbf599/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:31.422380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-e03968ab-03bc-4d6d-9805-ccd62e024dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e78427b641a415482dfbaabcd7812ab'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:31.431176 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1096/2208] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:31 2022] DELETE /identity/v3/users/8e78427b641a415482dfbaabcd7812ab => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:31.473374 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c2c5d-6c51-4135-86d3-96092ecb922b admin admin] RBAC: Authorizing `identity:create_grant(project_id=93d2fd56c8f142a38927ad957869c5d2, user_id=5b02b318056e4832aca3186795cbf599, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:31.486337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c2c5d-6c51-4135-86d3-96092ecb922b admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:31.524449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1113/2209] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:31 2022] PUT /identity/v3/projects/93d2fd56c8f142a38927ad957869c5d2/users/5b02b318056e4832aca3186795cbf599/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:31.543625 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f111ad-b7ca-4498-97ff-e2508ba1a606 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:31.543944 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f111ad-b7ca-4498-97ff-e2508ba1a606 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:31.544224 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f111ad-b7ca-4498-97ff-e2508ba1a606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:31.601237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c2c5d-6c51-4135-86d3-96092ecb922b admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:31.603970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-716c2c5d-6c51-4135-86d3-96092ecb922b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:31.622532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-e9f111ad-b7ca-4498-97ff-e2508ba1a606 None None] MFA Rules not processed for user `5b02b318056e4832aca3186795cbf599`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:31.623798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e9f111ad-b7ca-4498-97ff-e2508ba1a606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:31.767818 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c2c5d-6c51-4135-86d3-96092ecb922b admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:31.772286 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e796ef5-5fed-41af-b867-35092bccb8c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:31.773210 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e796ef5-5fed-41af-b867-35092bccb8c4 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:31.773653 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e796ef5-5fed-41af-b867-35092bccb8c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:31.774119 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e796ef5-5fed-41af-b867-35092bccb8c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:31.776373 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5e796ef5-5fed-41af-b867-35092bccb8c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:31.794292 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1097/2210] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:31.906707 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e796ef5-5fed-41af-b867-35092bccb8c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:31.908234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e796ef5-5fed-41af-b867-35092bccb8c4 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:31.925572 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f111ad-b7ca-4498-97ff-e2508ba1a606 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:31.925572 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e9f111ad-b7ca-4498-97ff-e2508ba1a606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:31.931030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1114/2211] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:31.934793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e796ef5-5fed-41af-b867-35092bccb8c4 service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:31.937628 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5e796ef5-5fed-41af-b867-35092bccb8c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:31.945220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f111ad-b7ca-4498-97ff-e2508ba1a606 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:31.953804 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb519e-bfca-4ff2-8c7f-ecabdc5f8217 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:31.955023 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb519e-bfca-4ff2-8c7f-ecabdc5f8217 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:31.955310 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb519e-bfca-4ff2-8c7f-ecabdc5f8217 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:31.955751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb519e-bfca-4ff2-8c7f-ecabdc5f8217 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:31.977810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb519e-bfca-4ff2-8c7f-ecabdc5f8217 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:31.978143 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb519e-bfca-4ff2-8c7f-ecabdc5f8217 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:32.004984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1098/2212] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:46:32.041078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb519e-bfca-4ff2-8c7f-ecabdc5f8217 service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:32.042612 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d0eb519e-bfca-4ff2-8c7f-ecabdc5f8217 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:32.069168 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e796ef5-5fed-41af-b867-35092bccb8c4 service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:32.078504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb519e-bfca-4ff2-8c7f-ecabdc5f8217 service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:32.078821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677bc7a-d067-4cb9-8c2e-90853f9a4ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:32.079877 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677bc7a-d067-4cb9-8c2e-90853f9a4ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:32.080168 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677bc7a-d067-4cb9-8c2e-90853f9a4ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:32.080446 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677bc7a-d067-4cb9-8c2e-90853f9a4ca0 admin admin] PATH_INFO: `/v3/projects/18f2f01634f34582992937d1beeaadf7` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:32.083483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677bc7a-d067-4cb9-8c2e-90853f9a4ca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18f2f01634f34582992937d1beeaadf7)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:32.091220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677bc7a-d067-4cb9-8c2e-90853f9a4ca0 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:32.101282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f74c94-e019-4158-94bb-7c336b218fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:32.102756 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f74c94-e019-4158-94bb-7c336b218fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:32.103177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f74c94-e019-4158-94bb-7c336b218fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:32.103712 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f74c94-e019-4158-94bb-7c336b218fe8 admin admin] PATH_INFO: `/v3/users/5b02b318056e4832aca3186795cbf599` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:32.109107 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f74c94-e019-4158-94bb-7c336b218fe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b02b318056e4832aca3186795cbf599)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:32.113780 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f74c94-e019-4158-94bb-7c336b218fe8 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:32.134997 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:32.166472 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:32.184810 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d677bc7a-d067-4cb9-8c2e-90853f9a4ca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18f2f01634f34582992937d1beeaadf7', 'request_id': 'req-d677bc7a-d067-4cb9-8c2e-90853f9a4ca0'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:32.200473 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d677bc7a-d067-4cb9-8c2e-90853f9a4ca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18f2f01634f34582992937d1beeaadf7', 'request_id': 'req-d677bc7a-d067-4cb9-8c2e-90853f9a4ca0'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:32.212872 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d677bc7a-d067-4cb9-8c2e-90853f9a4ca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18f2f01634f34582992937d1beeaadf7', 'request_id': 'req-d677bc7a-d067-4cb9-8c2e-90853f9a4ca0'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:32.238047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d677bc7a-d067-4cb9-8c2e-90853f9a4ca0 admin admin] The token cache is being invalidate because project 18f2f01634f34582992937d1beeaadf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:32.238764 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d677bc7a-d067-4cb9-8c2e-90853f9a4ca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:32.240217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d677bc7a-d067-4cb9-8c2e-90853f9a4ca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:32.243698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1115/2213] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:31 2022] DELETE /identity/v3/projects/18f2f01634f34582992937d1beeaadf7 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:32.255734 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f883408-73ff-4d0b-974e-8ae2101d801d None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:32.257012 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f883408-73ff-4d0b-974e-8ae2101d801d None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:32.258345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-68f74c94-e019-4158-94bb-7c336b218fe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b02b318056e4832aca3186795cbf599'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:32.258563 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f883408-73ff-4d0b-974e-8ae2101d801d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:32.307218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-68f74c94-e019-4158-94bb-7c336b218fe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b02b318056e4832aca3186795cbf599'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:32.307457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-5f883408-73ff-4d0b-974e-8ae2101d801d None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:32.311754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5f883408-73ff-4d0b-974e-8ae2101d801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:32.334778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-68f74c94-e019-4158-94bb-7c336b218fe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b02b318056e4832aca3186795cbf599'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:32.336705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-68f74c94-e019-4158-94bb-7c336b218fe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b02b318056e4832aca3186795cbf599'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:32.357226 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-68f74c94-e019-4158-94bb-7c336b218fe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b02b318056e4832aca3186795cbf599'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:32.370026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1099/2214] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:32 2022] DELETE /identity/v3/users/5b02b318056e4832aca3186795cbf599 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:32.372610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a60cbb-7242-4b13-9979-3dea842c3706 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:32.372914 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a60cbb-7242-4b13-9979-3dea842c3706 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:32.373199 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a60cbb-7242-4b13-9979-3dea842c3706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:32.403162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-88a60cbb-7242-4b13-9979-3dea842c3706 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:32.404760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-88a60cbb-7242-4b13-9979-3dea842c3706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:32.506528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1100/2215] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:32.519581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1116/2216] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:32.521816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a60cbb-7242-4b13-9979-3dea842c3706 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:32.523753 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-88a60cbb-7242-4b13-9979-3dea842c3706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:32.524005 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfb822-9a49-4b43-9edd-23dbdf46adf2 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:32.524298 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfb822-9a49-4b43-9edd-23dbdf46adf2 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:32.524868 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bfb822-9a49-4b43-9edd-23dbdf46adf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:32.545069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-91bfb822-9a49-4b43-9edd-23dbdf46adf2 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:32.547277 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-91bfb822-9a49-4b43-9edd-23dbdf46adf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:32.564147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1117/2217] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:32.570640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfb822-9a49-4b43-9edd-23dbdf46adf2 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:32.592294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bfb822-9a49-4b43-9edd-23dbdf46adf2 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:32.607812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6ca4a-2c76-4bec-99c8-29e05af5deae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:32.609006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6ca4a-2c76-4bec-99c8-29e05af5deae admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:32.610007 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6ca4a-2c76-4bec-99c8-29e05af5deae admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:32.614559 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6ca4a-2c76-4bec-99c8-29e05af5deae admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:32.617032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6ca4a-2c76-4bec-99c8-29e05af5deae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:46:32.619527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6ca4a-2c76-4bec-99c8-29e05af5deae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:32.628114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6ca4a-2c76-4bec-99c8-29e05af5deae admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:32.628946 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a60cbb-7242-4b13-9979-3dea842c3706 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:32.633955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06959c3-00f9-4081-8dc7-b74f294edc97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:32.634765 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06959c3-00f9-4081-8dc7-b74f294edc97 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:32.635161 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06959c3-00f9-4081-8dc7-b74f294edc97 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:32.635706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06959c3-00f9-4081-8dc7-b74f294edc97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:32.648810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06959c3-00f9-4081-8dc7-b74f294edc97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:32.650506 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06959c3-00f9-4081-8dc7-b74f294edc97 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:32.666088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1118/2218] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:46:32.667894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6ca4a-2c76-4bec-99c8-29e05af5deae admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:32.668971 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1101/2219] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:32.706419 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6ca4a-2c76-4bec-99c8-29e05af5deae admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:32.716428 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd50a5-3aa7-4f0b-bf66-900fad1cb3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:32.718174 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd50a5-3aa7-4f0b-bf66-900fad1cb3d3 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:32.718581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd50a5-3aa7-4f0b-bf66-900fad1cb3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:32.719069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd50a5-3aa7-4f0b-bf66-900fad1cb3d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:32.720245 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd50a5-3aa7-4f0b-bf66-900fad1cb3d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:32.723594 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd50a5-3aa7-4f0b-bf66-900fad1cb3d3 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:32.789015 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1119/2220] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:32 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:32.795129 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06959c3-00f9-4081-8dc7-b74f294edc97 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:32.816310 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06959c3-00f9-4081-8dc7-b74f294edc97 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:32.902279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:33.011486 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa236c4-5980-4ea2-a9c0-d8e41c3e7eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:33.014204 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa236c4-5980-4ea2-a9c0-d8e41c3e7eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:33.014800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa236c4-5980-4ea2-a9c0-d8e41c3e7eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:33.015287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa236c4-5980-4ea2-a9c0-d8e41c3e7eb9 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:33.020198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa236c4-5980-4ea2-a9c0-d8e41c3e7eb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:33.036492 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa236c4-5980-4ea2-a9c0-d8e41c3e7eb9 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:33.121402 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1102/2221] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:32 2022] POST /identity/v3/users => generated 288 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:33.122438 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd50a5-3aa7-4f0b-bf66-900fad1cb3d3 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:33.142105 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd50a5-3aa7-4f0b-bf66-900fad1cb3d3 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:33.155837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f1dc3-a072-4ed7-94a9-0312ae073838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:33.162240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f1dc3-a072-4ed7-94a9-0312ae073838 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:33.162972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f1dc3-a072-4ed7-94a9-0312ae073838 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:33.165914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f1dc3-a072-4ed7-94a9-0312ae073838 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:33.168644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f1dc3-a072-4ed7-94a9-0312ae073838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:33.172650 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f1dc3-a072-4ed7-94a9-0312ae073838 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:33.188699 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1120/2222] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:33.194614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa236c4-5980-4ea2-a9c0-d8e41c3e7eb9 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:33.208976 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f1dc3-a072-4ed7-94a9-0312ae073838 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:33.212446 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e65f1dc3-a072-4ed7-94a9-0312ae073838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:33.214319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa236c4-5980-4ea2-a9c0-d8e41c3e7eb9 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:33.225607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324bba8-d572-4d75-825a-b10901c807c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:33.226965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324bba8-d572-4d75-825a-b10901c807c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:33.227467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324bba8-d572-4d75-825a-b10901c807c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:33.228198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324bba8-d572-4d75-825a-b10901c807c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:33.229797 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324bba8-d572-4d75-825a-b10901c807c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:33.233010 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324bba8-d572-4d75-825a-b10901c807c9 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:33.274869 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1103/2223] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:33.277472 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324bba8-d572-4d75-825a-b10901c807c9 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:33.318341 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324bba8-d572-4d75-825a-b10901c807c9 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:33.332093 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10830c18-1910-458b-827c-2e4d7b8b2552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:33.334723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10830c18-1910-458b-827c-2e4d7b8b2552 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:33.336019 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10830c18-1910-458b-827c-2e4d7b8b2552 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:33.336573 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10830c18-1910-458b-827c-2e4d7b8b2552 admin admin] PATH_INFO: `/v3/projects/4829f904e10c4df197b253f1211157af/users/6cd3a653cb514e37bad8080d8a5cc7b0/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:33.370680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f1dc3-a072-4ed7-94a9-0312ae073838 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:33.385003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0850d8-d55b-4e8a-a3c6-a412bc717595 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:33.385808 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0850d8-d55b-4e8a-a3c6-a412bc717595 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:33.386105 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0850d8-d55b-4e8a-a3c6-a412bc717595 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:33.386390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0850d8-d55b-4e8a-a3c6-a412bc717595 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:33.395710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ef0850d8-d55b-4e8a-a3c6-a412bc717595 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:33.402143 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10830c18-1910-458b-827c-2e4d7b8b2552 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4829f904e10c4df197b253f1211157af, user_id=6cd3a653cb514e37bad8080d8a5cc7b0, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:33.410315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10830c18-1910-458b-827c-2e4d7b8b2552 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:33.436986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:33.455486 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1104/2224] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:33 2022] PUT /identity/v3/projects/4829f904e10c4df197b253f1211157af/users/6cd3a653cb514e37bad8080d8a5cc7b0/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:33.456604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10830c18-1910-458b-827c-2e4d7b8b2552 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:33.460364 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-10830c18-1910-458b-827c-2e4d7b8b2552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:33.565052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0850d8-d55b-4e8a-a3c6-a412bc717595 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:33.566522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0850d8-d55b-4e8a-a3c6-a412bc717595 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:33.577112 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1121/2225] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:33.577556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10830c18-1910-458b-827c-2e4d7b8b2552 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:33.578960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6daecc-d6ff-46cc-a2a9-bc68ce4ed179 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:33.579251 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6daecc-d6ff-46cc-a2a9-bc68ce4ed179 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:33.579565 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6daecc-d6ff-46cc-a2a9-bc68ce4ed179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:33.590065 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982df72d-358f-4da0-a3fd-173241fb7f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:33.591518 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982df72d-358f-4da0-a3fd-173241fb7f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:33.591966 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982df72d-358f-4da0-a3fd-173241fb7f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:33.592396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982df72d-358f-4da0-a3fd-173241fb7f98 admin admin] PATH_INFO: `/v3/projects/93d2fd56c8f142a38927ad957869c5d2` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:33.594862 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982df72d-358f-4da0-a3fd-173241fb7f98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93d2fd56c8f142a38927ad957869c5d2)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:33.598099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982df72d-358f-4da0-a3fd-173241fb7f98 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:33.622267 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-bb6daecc-d6ff-46cc-a2a9-bc68ce4ed179 None None] MFA Rules not processed for user `6cd3a653cb514e37bad8080d8a5cc7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:33.624461 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-bb6daecc-d6ff-46cc-a2a9-bc68ce4ed179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:33.690567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-982df72d-358f-4da0-a3fd-173241fb7f98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93d2fd56c8f142a38927ad957869c5d2', 'request_id': 'req-982df72d-358f-4da0-a3fd-173241fb7f98'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:33.700867 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-982df72d-358f-4da0-a3fd-173241fb7f98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93d2fd56c8f142a38927ad957869c5d2', 'request_id': 'req-982df72d-358f-4da0-a3fd-173241fb7f98'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:33.709529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-982df72d-358f-4da0-a3fd-173241fb7f98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93d2fd56c8f142a38927ad957869c5d2', 'request_id': 'req-982df72d-358f-4da0-a3fd-173241fb7f98'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:33.728620 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-982df72d-358f-4da0-a3fd-173241fb7f98 admin admin] The token cache is being invalidate because project 93d2fd56c8f142a38927ad957869c5d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:33.729215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-982df72d-358f-4da0-a3fd-173241fb7f98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:33.730323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-982df72d-358f-4da0-a3fd-173241fb7f98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:33.733208 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1105/2226] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:33 2022] DELETE /identity/v3/projects/93d2fd56c8f142a38927ad957869c5d2 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:33.740311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a1961-8001-4a87-91dd-4d35b999304f None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:33.740748 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a1961-8001-4a87-91dd-4d35b999304f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:33.742552 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91a1961-8001-4a87-91dd-4d35b999304f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:33.773179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1122/2227] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:33.786003 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c91a1961-8001-4a87-91dd-4d35b999304f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:33.787395 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c91a1961-8001-4a87-91dd-4d35b999304f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:33.795132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6daecc-d6ff-46cc-a2a9-bc68ce4ed179 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:33.799457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-bb6daecc-d6ff-46cc-a2a9-bc68ce4ed179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:33.804729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:33.895594 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6daecc-d6ff-46cc-a2a9-bc68ce4ed179 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:33.904563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1106/2228] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:33.916408 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfaa6ee-add2-4bc0-aafd-c983ea162409 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:33.916827 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfaa6ee-add2-4bc0-aafd-c983ea162409 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:33.917471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfaa6ee-add2-4bc0-aafd-c983ea162409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:33.942857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-bbfaa6ee-add2-4bc0-aafd-c983ea162409 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:33.944122 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-bbfaa6ee-add2-4bc0-aafd-c983ea162409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:33.956217 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1107/2229] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:33.960350 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfaa6ee-add2-4bc0-aafd-c983ea162409 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:33.962038 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-bbfaa6ee-add2-4bc0-aafd-c983ea162409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:33.975568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfaa6ee-add2-4bc0-aafd-c983ea162409 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:33.976743 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a76b9e-e114-4a7f-b702-2ae341742f3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:33.977870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a76b9e-e114-4a7f-b702-2ae341742f3c service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:33.978352 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a76b9e-e114-4a7f-b702-2ae341742f3c service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:33.978880 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a76b9e-e114-4a7f-b702-2ae341742f3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:33.980591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee71de-d6b3-4c8a-aabf-e4481035c232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:33.981457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee71de-d6b3-4c8a-aabf-e4481035c232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:33.981732 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee71de-d6b3-4c8a-aabf-e4481035c232 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:33.982001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee71de-d6b3-4c8a-aabf-e4481035c232 admin admin] PATH_INFO: `/v3/users/b6c69ec246b442d0b8be70425254a42f` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:33.983223 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee71de-d6b3-4c8a-aabf-e4481035c232 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6c69ec246b442d0b8be70425254a42f)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:33.986366 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee71de-d6b3-4c8a-aabf-e4481035c232 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:33.991858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:34.055405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-2cee71de-d6b3-4c8a-aabf-e4481035c232 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6c69ec246b442d0b8be70425254a42f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:34.073370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a76b9e-e114-4a7f-b702-2ae341742f3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:34.075286 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a76b9e-e114-4a7f-b702-2ae341742f3c service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:34.079724 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-2cee71de-d6b3-4c8a-aabf-e4481035c232 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6c69ec246b442d0b8be70425254a42f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:34.089235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1123/2230] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:46:34.090435 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a76b9e-e114-4a7f-b702-2ae341742f3c service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:34.091720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-2cee71de-d6b3-4c8a-aabf-e4481035c232 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6c69ec246b442d0b8be70425254a42f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:34.092632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-2cee71de-d6b3-4c8a-aabf-e4481035c232 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6c69ec246b442d0b8be70425254a42f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:34.103930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-2cee71de-d6b3-4c8a-aabf-e4481035c232 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6c69ec246b442d0b8be70425254a42f'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:34.107842 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a76b9e-e114-4a7f-b702-2ae341742f3c service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:34.113414 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1108/2231] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:33 2022] DELETE /identity/v3/users/b6c69ec246b442d0b8be70425254a42f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:34.115964 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f7fd1-b8cc-4dd2-8602-fb6bbb2f038b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:34.117332 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f7fd1-b8cc-4dd2-8602-fb6bbb2f038b admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:34.117909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f7fd1-b8cc-4dd2-8602-fb6bbb2f038b admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:34.118364 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f7fd1-b8cc-4dd2-8602-fb6bbb2f038b admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:34.118857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f3a08-949a-4723-b7a7-d20693ce6605 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:34.119220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f3a08-949a-4723-b7a7-d20693ce6605 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:34.119595 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f3a08-949a-4723-b7a7-d20693ce6605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:34.119887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f7fd1-b8cc-4dd2-8602-fb6bbb2f038b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:46:34.120409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f7fd1-b8cc-4dd2-8602-fb6bbb2f038b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:34.122652 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f7fd1-b8cc-4dd2-8602-fb6bbb2f038b admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:34.142904 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-e73f3a08-949a-4723-b7a7-d20693ce6605 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:34.144443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e73f3a08-949a-4723-b7a7-d20693ce6605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:34.147978 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1124/2232] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:46:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:46:34.152972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f7fd1-b8cc-4dd2-8602-fb6bbb2f038b admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:34.157471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-4b2f7fd1-b8cc-4dd2-8602-fb6bbb2f038b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:34.176259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f7fd1-b8cc-4dd2-8602-fb6bbb2f038b admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:34.255435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1109/2233] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:34.257295 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30f5d1-4ff6-4f79-9882-85a1606f6c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:34.258697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30f5d1-4ff6-4f79-9882-85a1606f6c96 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:34.259061 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30f5d1-4ff6-4f79-9882-85a1606f6c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:34.259440 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30f5d1-4ff6-4f79-9882-85a1606f6c96 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:34.260211 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30f5d1-4ff6-4f79-9882-85a1606f6c96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:34.262579 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30f5d1-4ff6-4f79-9882-85a1606f6c96 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:34.270966 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f3a08-949a-4723-b7a7-d20693ce6605 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:34.273558 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e73f3a08-949a-4723-b7a7-d20693ce6605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:34.300052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1125/2234] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:34 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:34.304895 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30f5d1-4ff6-4f79-9882-85a1606f6c96 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:34.316429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:34.366271 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f3a08-949a-4723-b7a7-d20693ce6605 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:34.406272 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30f5d1-4ff6-4f79-9882-85a1606f6c96 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:34.413024 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908b84e-aaf8-4ab1-b73d-a765baaff13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:34.413808 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908b84e-aaf8-4ab1-b73d-a765baaff13d admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:34.414089 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908b84e-aaf8-4ab1-b73d-a765baaff13d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:34.414599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908b84e-aaf8-4ab1-b73d-a765baaff13d admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:34.415179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908b84e-aaf8-4ab1-b73d-a765baaff13d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:34.417189 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908b84e-aaf8-4ab1-b73d-a765baaff13d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:34.436120 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb0863-d34a-40da-bfe4-6fad6b7948a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:34.436447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb0863-d34a-40da-bfe4-6fad6b7948a8 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:34.437037 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb0863-d34a-40da-bfe4-6fad6b7948a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:34.437037 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb0863-d34a-40da-bfe4-6fad6b7948a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:34.443250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1126/2235] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:34 2022] POST /identity/v3/users => generated 288 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:34.447457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908b84e-aaf8-4ab1-b73d-a765baaff13d admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:34.449706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb0863-d34a-40da-bfe4-6fad6b7948a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:34.451790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb0863-d34a-40da-bfe4-6fad6b7948a8 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:34.461988 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908b84e-aaf8-4ab1-b73d-a765baaff13d admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:34.467141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3298092-aeb8-47e6-84bf-da64ddb8ce59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:34.467858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3298092-aeb8-47e6-84bf-da64ddb8ce59 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:34.468141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3298092-aeb8-47e6-84bf-da64ddb8ce59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:34.468536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3298092-aeb8-47e6-84bf-da64ddb8ce59 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:34.470011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3298092-aeb8-47e6-84bf-da64ddb8ce59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:34.472310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3298092-aeb8-47e6-84bf-da64ddb8ce59 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:34.478123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1110/2236] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:46:34.478261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:34.489931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1127/2237] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:46:34.492387 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb0863-d34a-40da-bfe4-6fad6b7948a8 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:34.511051 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb0863-d34a-40da-bfe4-6fad6b7948a8 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:34.516763 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83046359-bfb2-4da5-955a-53a5fc5affbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:34.517780 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83046359-bfb2-4da5-955a-53a5fc5affbc admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:34.518060 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83046359-bfb2-4da5-955a-53a5fc5affbc admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:34.518391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83046359-bfb2-4da5-955a-53a5fc5affbc admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:34.519908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83046359-bfb2-4da5-955a-53a5fc5affbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:34.520964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83046359-bfb2-4da5-955a-53a5fc5affbc admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:34.535541 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1111/2238] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:34.537611 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3298092-aeb8-47e6-84bf-da64ddb8ce59 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:34.563717 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3298092-aeb8-47e6-84bf-da64ddb8ce59 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:34.580662 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7e065-3319-4474-b759-171ed28ab9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:34.581574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7e065-3319-4474-b759-171ed28ab9f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:34.581858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7e065-3319-4474-b759-171ed28ab9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:34.582145 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7e065-3319-4474-b759-171ed28ab9f8 admin admin] PATH_INFO: `/v3/projects/7ea4f3275fab42338617fe09eafdd143/users/7fcf899d893b4122878db3365f9ef120/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:34.618640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7e065-3319-4474-b759-171ed28ab9f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ea4f3275fab42338617fe09eafdd143, user_id=7fcf899d893b4122878db3365f9ef120, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:34.622025 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7e065-3319-4474-b759-171ed28ab9f8 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:34.640453 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1128/2239] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:34 2022] PUT /identity/v3/projects/7ea4f3275fab42338617fe09eafdd143/users/7fcf899d893b4122878db3365f9ef120/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:34.651862 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3b45c-5780-4c86-964f-af1e780c7068 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:34.652383 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3b45c-5780-4c86-964f-af1e780c7068 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:34.652664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3b45c-5780-4c86-964f-af1e780c7068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:34.681960 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-e5a3b45c-5780-4c86-964f-af1e780c7068 None None] MFA Rules not processed for user `7fcf899d893b4122878db3365f9ef120`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:34.683587 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e5a3b45c-5780-4c86-964f-af1e780c7068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:34.793785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1112/2240] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:34.809599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7e065-3319-4474-b759-171ed28ab9f8 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:34.816901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-2cb7e065-3319-4474-b759-171ed28ab9f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:34.872711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3b45c-5780-4c86-964f-af1e780c7068 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:34.874712 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e5a3b45c-5780-4c86-964f-af1e780c7068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:34.943440 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7e065-3319-4474-b759-171ed28ab9f8 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:34.948106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873bdeea-c8b6-4752-b560-8635b0baffd4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ed133721ed044708ee362c93f564d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:34.950403 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bdeea-c8b6-4752-b560-8635b0baffd4 service glance] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:34.950698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bdeea-c8b6-4752-b560-8635b0baffd4 service glance] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:34.950971 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bdeea-c8b6-4752-b560-8635b0baffd4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:34.962482 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873bdeea-c8b6-4752-b560-8635b0baffd4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:34.964116 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873bdeea-c8b6-4752-b560-8635b0baffd4 service glance] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:34.976237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1129/2241] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:35.003549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3b45c-5780-4c86-964f-af1e780c7068 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:35.008349 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c75c4-5760-4fad-be31-c359a9dd7ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:35.009229 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c75c4-5760-4fad-be31-c359a9dd7ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:35.009741 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c75c4-5760-4fad-be31-c359a9dd7ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:35.010108 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c75c4-5760-4fad-be31-c359a9dd7ad5 admin admin] PATH_INFO: `/v3/projects/8924f21d5e90417393b32c64dd1fbb81` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:35.011879 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c75c4-5760-4fad-be31-c359a9dd7ad5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8924f21d5e90417393b32c64dd1fbb81)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:35.013824 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c75c4-5760-4fad-be31-c359a9dd7ad5 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:35.017075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:35.104822 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-3f5c75c4-5760-4fad-be31-c359a9dd7ad5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8924f21d5e90417393b32c64dd1fbb81', 'request_id': 'req-3f5c75c4-5760-4fad-be31-c359a9dd7ad5'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:35.119679 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-3f5c75c4-5760-4fad-be31-c359a9dd7ad5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8924f21d5e90417393b32c64dd1fbb81', 'request_id': 'req-3f5c75c4-5760-4fad-be31-c359a9dd7ad5'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:35.153617 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-3f5c75c4-5760-4fad-be31-c359a9dd7ad5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8924f21d5e90417393b32c64dd1fbb81', 'request_id': 'req-3f5c75c4-5760-4fad-be31-c359a9dd7ad5'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:35.179214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-3f5c75c4-5760-4fad-be31-c359a9dd7ad5 admin admin] The token cache is being invalidate because project 8924f21d5e90417393b32c64dd1fbb81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:35.179984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-3f5c75c4-5760-4fad-be31-c359a9dd7ad5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:35.191610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-3f5c75c4-5760-4fad-be31-c359a9dd7ad5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:35.194942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1113/2242] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:34 2022] DELETE /identity/v3/projects/8924f21d5e90417393b32c64dd1fbb81 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:35.198859 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873bdeea-c8b6-4752-b560-8635b0baffd4 service glance] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:35.243521 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873bdeea-c8b6-4752-b560-8635b0baffd4 service glance] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:35.334522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1420c6-224a-4f9b-80cd-d8a62db93177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:35.336874 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1420c6-224a-4f9b-80cd-d8a62db93177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:35.336874 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1420c6-224a-4f9b-80cd-d8a62db93177 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:35.336874 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1420c6-224a-4f9b-80cd-d8a62db93177 admin admin] PATH_INFO: `/v3/users/01885b8efaff48ce8ca3c5ce58a8d955` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:35.337567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1420c6-224a-4f9b-80cd-d8a62db93177 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01885b8efaff48ce8ca3c5ce58a8d955)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:35.339747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1420c6-224a-4f9b-80cd-d8a62db93177 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:35.409915 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8d1420c6-224a-4f9b-80cd-d8a62db93177 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01885b8efaff48ce8ca3c5ce58a8d955'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:35.434649 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8d1420c6-224a-4f9b-80cd-d8a62db93177 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01885b8efaff48ce8ca3c5ce58a8d955'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:35.448444 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8d1420c6-224a-4f9b-80cd-d8a62db93177 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01885b8efaff48ce8ca3c5ce58a8d955'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:35.461124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8d1420c6-224a-4f9b-80cd-d8a62db93177 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01885b8efaff48ce8ca3c5ce58a8d955'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:35.462234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8d1420c6-224a-4f9b-80cd-d8a62db93177 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01885b8efaff48ce8ca3c5ce58a8d955'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:35.464374 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:35.471178 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1130/2243] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:35 2022] DELETE /identity/v3/users/01885b8efaff48ce8ca3c5ce58a8d955 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:35.621601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819da42-2fad-4768-8038-d2888788cdb3 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:35.621983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819da42-2fad-4768-8038-d2888788cdb3 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:35.622171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819da42-2fad-4768-8038-d2888788cdb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:35.645475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-1819da42-2fad-4768-8038-d2888788cdb3 None None] MFA Rules not processed for user `6cd3a653cb514e37bad8080d8a5cc7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:35.646794 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-1819da42-2fad-4768-8038-d2888788cdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:35.723483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1114/2244] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:35.897531 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1420c6-224a-4f9b-80cd-d8a62db93177 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:35.900308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-8d1420c6-224a-4f9b-80cd-d8a62db93177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:35.986570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1420c6-224a-4f9b-80cd-d8a62db93177 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:35.993108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b07ce0-542e-42de-b433-4a4609385253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:35.994748 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b07ce0-542e-42de-b433-4a4609385253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:35.995049 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b07ce0-542e-42de-b433-4a4609385253 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:35.995417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b07ce0-542e-42de-b433-4a4609385253 admin admin] PATH_INFO: `/v3/projects/e8da91a7e1f3489e979e4a14154ed054` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:35.996708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b07ce0-542e-42de-b433-4a4609385253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8da91a7e1f3489e979e4a14154ed054)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:35.998714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b07ce0-542e-42de-b433-4a4609385253 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:36.062389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-49b07ce0-542e-42de-b433-4a4609385253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8da91a7e1f3489e979e4a14154ed054', 'request_id': 'req-49b07ce0-542e-42de-b433-4a4609385253'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:36.069271 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-49b07ce0-542e-42de-b433-4a4609385253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8da91a7e1f3489e979e4a14154ed054', 'request_id': 'req-49b07ce0-542e-42de-b433-4a4609385253'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:36.078940 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-49b07ce0-542e-42de-b433-4a4609385253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8da91a7e1f3489e979e4a14154ed054', 'request_id': 'req-49b07ce0-542e-42de-b433-4a4609385253'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:36.091857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-49b07ce0-542e-42de-b433-4a4609385253 admin admin] The token cache is being invalidate because project e8da91a7e1f3489e979e4a14154ed054 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:36.092237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-49b07ce0-542e-42de-b433-4a4609385253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:36.096065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-49b07ce0-542e-42de-b433-4a4609385253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:36.099279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1131/2245] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:35 2022] DELETE /identity/v3/projects/e8da91a7e1f3489e979e4a14154ed054 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:36.108773 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdf777-4a53-4564-9680-6db5547058dc None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:36.109335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdf777-4a53-4564-9680-6db5547058dc None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:36.109430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdf777-4a53-4564-9680-6db5547058dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:36.134767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c9bdf777-4a53-4564-9680-6db5547058dc None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:36.136025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c9bdf777-4a53-4564-9680-6db5547058dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:36.224731 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1115/2246] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:36.233103 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f264d-0bfe-4ab4-9563-50a945dff341 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:36.234041 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f264d-0bfe-4ab4-9563-50a945dff341 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:36.235262 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f264d-0bfe-4ab4-9563-50a945dff341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:36.268700 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-057f264d-0bfe-4ab4-9563-50a945dff341 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:36.270959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-057f264d-0bfe-4ab4-9563-50a945dff341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:36.285428 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1132/2247] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:36.292019 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdf777-4a53-4564-9680-6db5547058dc None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:36.325553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdf777-4a53-4564-9680-6db5547058dc None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:36.347709 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b3737-6e7c-43a4-b2f6-16fe7c080f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:36.350675 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b3737-6e7c-43a4-b2f6-16fe7c080f39 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:36.351682 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b3737-6e7c-43a4-b2f6-16fe7c080f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:36.352497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b3737-6e7c-43a4-b2f6-16fe7c080f39 admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:36.354022 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b3737-6e7c-43a4-b2f6-16fe7c080f39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:46:36.354620 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b3737-6e7c-43a4-b2f6-16fe7c080f39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:36.357076 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b3737-6e7c-43a4-b2f6-16fe7c080f39 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:36.393474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1116/2248] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:46:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:46:36.394828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f264d-0bfe-4ab4-9563-50a945dff341 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:36.415763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f264d-0bfe-4ab4-9563-50a945dff341 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:36.422036 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8458f-ffde-4563-a334-01e535a5e2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:36.422656 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8458f-ffde-4563-a334-01e535a5e2f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:36.423118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8458f-ffde-4563-a334-01e535a5e2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:36.423558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8458f-ffde-4563-a334-01e535a5e2f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:36.424260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8458f-ffde-4563-a334-01e535a5e2f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:36.426618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8458f-ffde-4563-a334-01e535a5e2f5 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:36.464326 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1133/2249] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:36 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:36.465105 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b3737-6e7c-43a4-b2f6-16fe7c080f39 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:36.484022 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b3737-6e7c-43a4-b2f6-16fe7c080f39 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:36.522929 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:36.571496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc542e-a5d4-428a-8dec-563edddb4bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:36.572263 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc542e-a5d4-428a-8dec-563edddb4bdf admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:36.572623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc542e-a5d4-428a-8dec-563edddb4bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:36.573411 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc542e-a5d4-428a-8dec-563edddb4bdf admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:36.573773 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc542e-a5d4-428a-8dec-563edddb4bdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:36.576945 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc542e-a5d4-428a-8dec-563edddb4bdf admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:36.604446 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1117/2250] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:36 2022] POST /identity/v3/users => generated 286 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:36.608725 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8458f-ffde-4563-a334-01e535a5e2f5 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:36.626261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8458f-ffde-4563-a334-01e535a5e2f5 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:36.631375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d066bcc-0a05-4b76-87bd-c8e79970c10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:36.632182 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d066bcc-0a05-4b76-87bd-c8e79970c10d admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:36.632313 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d066bcc-0a05-4b76-87bd-c8e79970c10d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:36.632599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d066bcc-0a05-4b76-87bd-c8e79970c10d admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:36.633402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d066bcc-0a05-4b76-87bd-c8e79970c10d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:36.635073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d066bcc-0a05-4b76-87bd-c8e79970c10d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:36.652604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1134/2251] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:36.657515 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc542e-a5d4-428a-8dec-563edddb4bdf admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:36.669268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc542e-a5d4-428a-8dec-563edddb4bdf admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:36.674942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae86c3-02ed-4496-bf05-8826554e3559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:36.675674 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae86c3-02ed-4496-bf05-8826554e3559 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:36.676244 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae86c3-02ed-4496-bf05-8826554e3559 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:36.676528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae86c3-02ed-4496-bf05-8826554e3559 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:36.677266 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae86c3-02ed-4496-bf05-8826554e3559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:36.678961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae86c3-02ed-4496-bf05-8826554e3559 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:36.694828 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1118/2252] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:36.699799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d066bcc-0a05-4b76-87bd-c8e79970c10d admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:36.711454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d066bcc-0a05-4b76-87bd-c8e79970c10d admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:36.719669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9693c-3083-45ff-b010-0e2a39918d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:36.720671 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9693c-3083-45ff-b010-0e2a39918d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:36.721083 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9693c-3083-45ff-b010-0e2a39918d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:36.721453 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9693c-3083-45ff-b010-0e2a39918d53 admin admin] PATH_INFO: `/v3/projects/d0de877c39ed464fb198fc135518407c/users/1123290f06914897b19017b49302e807/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:36.741388 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9693c-3083-45ff-b010-0e2a39918d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0de877c39ed464fb198fc135518407c, user_id=1123290f06914897b19017b49302e807, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:36.744382 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9693c-3083-45ff-b010-0e2a39918d53 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:36.764474 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1135/2253] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:36 2022] PUT /identity/v3/projects/d0de877c39ed464fb198fc135518407c/users/1123290f06914897b19017b49302e807/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:36.779055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23dfb6-914b-41c4-af59-2840117c7c00 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:36.779290 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23dfb6-914b-41c4-af59-2840117c7c00 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:36.779652 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23dfb6-914b-41c4-af59-2840117c7c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:36.811693 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-6d23dfb6-914b-41c4-af59-2840117c7c00 None None] MFA Rules not processed for user `1123290f06914897b19017b49302e807`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:36.813478 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-6d23dfb6-914b-41c4-af59-2840117c7c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:36.885544 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1119/2254] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:36.898742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9693c-3083-45ff-b010-0e2a39918d53 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:36.900511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ccf9693c-3083-45ff-b010-0e2a39918d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:36.935414 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:36.973707 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9693c-3083-45ff-b010-0e2a39918d53 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:36.978239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811172ca-0d33-49f3-a73b-4bc638ab0ca6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ed133721ed044708ee362c93f564d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:36.978902 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811172ca-0d33-49f3-a73b-4bc638ab0ca6 service glance] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:36.979185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811172ca-0d33-49f3-a73b-4bc638ab0ca6 service glance] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:36.979496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811172ca-0d33-49f3-a73b-4bc638ab0ca6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:36.988664 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811172ca-0d33-49f3-a73b-4bc638ab0ca6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:36.990381 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811172ca-0d33-49f3-a73b-4bc638ab0ca6 service glance] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:37.012291 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1136/2255] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:39.373150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23dfb6-914b-41c4-af59-2840117c7c00 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:39.376910 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-6d23dfb6-914b-41c4-af59-2840117c7c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:39.533307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23dfb6-914b-41c4-af59-2840117c7c00 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:39.539203 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0268b6-33e0-4e64-9af1-c48b89fd2ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:39.540311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0268b6-33e0-4e64-9af1-c48b89fd2ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:39.540609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0268b6-33e0-4e64-9af1-c48b89fd2ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:39.540971 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0268b6-33e0-4e64-9af1-c48b89fd2ab8 admin admin] PATH_INFO: `/v3/users/b6dee91fadde4caf950245a63f8b0cb2` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:39.542635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0268b6-33e0-4e64-9af1-c48b89fd2ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6dee91fadde4caf950245a63f8b0cb2)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:39.548007 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0268b6-33e0-4e64-9af1-c48b89fd2ab8 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:39.633465 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:39.659497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-7a0268b6-33e0-4e64-9af1-c48b89fd2ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6dee91fadde4caf950245a63f8b0cb2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:39.682391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-7a0268b6-33e0-4e64-9af1-c48b89fd2ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6dee91fadde4caf950245a63f8b0cb2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:39.695238 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-7a0268b6-33e0-4e64-9af1-c48b89fd2ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6dee91fadde4caf950245a63f8b0cb2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:39.696268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-7a0268b6-33e0-4e64-9af1-c48b89fd2ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6dee91fadde4caf950245a63f8b0cb2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:39.718327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-7a0268b6-33e0-4e64-9af1-c48b89fd2ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6dee91fadde4caf950245a63f8b0cb2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:39.735495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1120/2256] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:39 2022] DELETE /identity/v3/users/b6dee91fadde4caf950245a63f8b0cb2 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:39.740761 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fb2fc-4659-40cd-ac85-5a5392dbdc0d None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:39.742619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fb2fc-4659-40cd-ac85-5a5392dbdc0d None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:39.743411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fb2fc-4659-40cd-ac85-5a5392dbdc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:39.801768 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-818fb2fc-4659-40cd-ac85-5a5392dbdc0d None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:39.806323 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-818fb2fc-4659-40cd-ac85-5a5392dbdc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:39.951438 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1137/2257] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:39.959544 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0268b6-33e0-4e64-9af1-c48b89fd2ab8 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:39.961025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-7a0268b6-33e0-4e64-9af1-c48b89fd2ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:40.155958 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0268b6-33e0-4e64-9af1-c48b89fd2ab8 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:40.179242 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db727231-9a20-4395-add2-ee760083eb5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:40.180133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db727231-9a20-4395-add2-ee760083eb5e service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:40.180417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db727231-9a20-4395-add2-ee760083eb5e service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:40.180700 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db727231-9a20-4395-add2-ee760083eb5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:40.207724 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db727231-9a20-4395-add2-ee760083eb5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:40.210015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db727231-9a20-4395-add2-ee760083eb5e service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:40.278404 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1121/2258] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:40.746907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818fb2fc-4659-40cd-ac85-5a5392dbdc0d None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:40.749367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-818fb2fc-4659-40cd-ac85-5a5392dbdc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:40.769583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818fb2fc-4659-40cd-ac85-5a5392dbdc0d None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:40.781993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a0316-63a4-4e8f-91e5-ac67e281f5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:40.783268 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a0316-63a4-4e8f-91e5-ac67e281f5af admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:40.783815 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a0316-63a4-4e8f-91e5-ac67e281f5af admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:40.784101 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a0316-63a4-4e8f-91e5-ac67e281f5af admin admin] PATH_INFO: `/v3/projects/81d61acad188428dade1ddc886ce57ee` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:40.786137 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a0316-63a4-4e8f-91e5-ac67e281f5af admin admin] RBAC: Authorizing `identity:delete_project(project_id=81d61acad188428dade1ddc886ce57ee)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:40.788339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a0316-63a4-4e8f-91e5-ac67e281f5af admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:40.857392 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-af4a0316-63a4-4e8f-91e5-ac67e281f5af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81d61acad188428dade1ddc886ce57ee', 'request_id': 'req-af4a0316-63a4-4e8f-91e5-ac67e281f5af'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:40.866165 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-af4a0316-63a4-4e8f-91e5-ac67e281f5af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81d61acad188428dade1ddc886ce57ee', 'request_id': 'req-af4a0316-63a4-4e8f-91e5-ac67e281f5af'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:40.878979 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-af4a0316-63a4-4e8f-91e5-ac67e281f5af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81d61acad188428dade1ddc886ce57ee', 'request_id': 'req-af4a0316-63a4-4e8f-91e5-ac67e281f5af'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:40.900368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-af4a0316-63a4-4e8f-91e5-ac67e281f5af admin admin] The token cache is being invalidate because project 81d61acad188428dade1ddc886ce57ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:40.900849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-af4a0316-63a4-4e8f-91e5-ac67e281f5af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:40.901872 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-af4a0316-63a4-4e8f-91e5-ac67e281f5af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:40.904417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1138/2259] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:40 2022] DELETE /identity/v3/projects/81d61acad188428dade1ddc886ce57ee => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:40.910493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db727231-9a20-4395-add2-ee760083eb5e service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:40.922764 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db727231-9a20-4395-add2-ee760083eb5e service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:40.998831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b99ab-f539-49fb-abe1-9423c2bc43ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:40.999821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b99ab-f539-49fb-abe1-9423c2bc43ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:41.000156 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b99ab-f539-49fb-abe1-9423c2bc43ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:41.000442 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b99ab-f539-49fb-abe1-9423c2bc43ef admin admin] PATH_INFO: `/v3/users/e6b23ddcff6e45359b46007698d402f6` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:41.001690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b99ab-f539-49fb-abe1-9423c2bc43ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6b23ddcff6e45359b46007698d402f6)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:41.003746 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b99ab-f539-49fb-abe1-9423c2bc43ef admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:41.084010 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a18b99ab-f539-49fb-abe1-9423c2bc43ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6b23ddcff6e45359b46007698d402f6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:41.105517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:41.122704 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a18b99ab-f539-49fb-abe1-9423c2bc43ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6b23ddcff6e45359b46007698d402f6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:41.135542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a18b99ab-f539-49fb-abe1-9423c2bc43ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6b23ddcff6e45359b46007698d402f6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:41.136825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a18b99ab-f539-49fb-abe1-9423c2bc43ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6b23ddcff6e45359b46007698d402f6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:41.149116 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a18b99ab-f539-49fb-abe1-9423c2bc43ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6b23ddcff6e45359b46007698d402f6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:41.166262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1122/2260] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:40 2022] DELETE /identity/v3/users/e6b23ddcff6e45359b46007698d402f6 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:41.570717 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a0316-63a4-4e8f-91e5-ac67e281f5af admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:41.575760 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-af4a0316-63a4-4e8f-91e5-ac67e281f5af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:41.598796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a0316-63a4-4e8f-91e5-ac67e281f5af admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:41.667178 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:41.683021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05445062-a735-4235-9ad9-cdc73112693c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:41.684595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05445062-a735-4235-9ad9-cdc73112693c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:41.684938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05445062-a735-4235-9ad9-cdc73112693c admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:41.685497 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05445062-a735-4235-9ad9-cdc73112693c admin admin] PATH_INFO: `/v3/projects/bee61c6de6b04cca9b0d984702e337cd` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:41.686737 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05445062-a735-4235-9ad9-cdc73112693c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bee61c6de6b04cca9b0d984702e337cd)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:41.689715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05445062-a735-4235-9ad9-cdc73112693c admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:41.761081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-05445062-a735-4235-9ad9-cdc73112693c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bee61c6de6b04cca9b0d984702e337cd', 'request_id': 'req-05445062-a735-4235-9ad9-cdc73112693c'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:41.774227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-05445062-a735-4235-9ad9-cdc73112693c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bee61c6de6b04cca9b0d984702e337cd', 'request_id': 'req-05445062-a735-4235-9ad9-cdc73112693c'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:41.785385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-05445062-a735-4235-9ad9-cdc73112693c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bee61c6de6b04cca9b0d984702e337cd', 'request_id': 'req-05445062-a735-4235-9ad9-cdc73112693c'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:41.799047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-05445062-a735-4235-9ad9-cdc73112693c admin admin] The token cache is being invalidate because project bee61c6de6b04cca9b0d984702e337cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:41.799389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-05445062-a735-4235-9ad9-cdc73112693c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:41.800135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-05445062-a735-4235-9ad9-cdc73112693c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:41.801756 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1139/2261] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:41 2022] DELETE /identity/v3/projects/bee61c6de6b04cca9b0d984702e337cd => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:41.808213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b99ab-f539-49fb-abe1-9423c2bc43ef admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:41.810833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a18b99ab-f539-49fb-abe1-9423c2bc43ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:41.835150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b99ab-f539-49fb-abe1-9423c2bc43ef admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:41.930484 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd61c0b-81ce-4a64-998a-2d7414a35a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:41.932136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd61c0b-81ce-4a64-998a-2d7414a35a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:41.932630 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd61c0b-81ce-4a64-998a-2d7414a35a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:41.933105 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd61c0b-81ce-4a64-998a-2d7414a35a3b admin admin] PATH_INFO: `/v3/users/34eb330cbb364212afe626f8a62dd2f4` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:41.935729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd61c0b-81ce-4a64-998a-2d7414a35a3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=34eb330cbb364212afe626f8a62dd2f4)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:41.940243 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd61c0b-81ce-4a64-998a-2d7414a35a3b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:42.021406 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4dd61c0b-81ce-4a64-998a-2d7414a35a3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34eb330cbb364212afe626f8a62dd2f4'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:42.035770 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4dd61c0b-81ce-4a64-998a-2d7414a35a3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34eb330cbb364212afe626f8a62dd2f4'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:42.044101 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4dd61c0b-81ce-4a64-998a-2d7414a35a3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34eb330cbb364212afe626f8a62dd2f4'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:42.045127 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4dd61c0b-81ce-4a64-998a-2d7414a35a3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34eb330cbb364212afe626f8a62dd2f4'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:42.058366 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4dd61c0b-81ce-4a64-998a-2d7414a35a3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34eb330cbb364212afe626f8a62dd2f4'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:42.069379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1123/2262] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:41 2022] DELETE /identity/v3/users/34eb330cbb364212afe626f8a62dd2f4 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:42.684257 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05445062-a735-4235-9ad9-cdc73112693c admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:42.688440 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-05445062-a735-4235-9ad9-cdc73112693c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:42.716138 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05445062-a735-4235-9ad9-cdc73112693c admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:42.839904 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac0e5a-09ea-47cd-b3f8-2116ef56f1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:42.842232 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac0e5a-09ea-47cd-b3f8-2116ef56f1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:42.843108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac0e5a-09ea-47cd-b3f8-2116ef56f1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:42.844216 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac0e5a-09ea-47cd-b3f8-2116ef56f1c9 admin admin] PATH_INFO: `/v3/projects/bd98346cd7634c1195be359c27172368` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:42.847876 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac0e5a-09ea-47cd-b3f8-2116ef56f1c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd98346cd7634c1195be359c27172368)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:42.849966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac0e5a-09ea-47cd-b3f8-2116ef56f1c9 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:42.938306 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-bfac0e5a-09ea-47cd-b3f8-2116ef56f1c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd98346cd7634c1195be359c27172368', 'request_id': 'req-bfac0e5a-09ea-47cd-b3f8-2116ef56f1c9'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:42.944333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-bfac0e5a-09ea-47cd-b3f8-2116ef56f1c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd98346cd7634c1195be359c27172368', 'request_id': 'req-bfac0e5a-09ea-47cd-b3f8-2116ef56f1c9'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:42.963012 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-bfac0e5a-09ea-47cd-b3f8-2116ef56f1c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd98346cd7634c1195be359c27172368', 'request_id': 'req-bfac0e5a-09ea-47cd-b3f8-2116ef56f1c9'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:42.989954 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-bfac0e5a-09ea-47cd-b3f8-2116ef56f1c9 admin admin] The token cache is being invalidate because project bd98346cd7634c1195be359c27172368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:42.990272 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-bfac0e5a-09ea-47cd-b3f8-2116ef56f1c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:42.990972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-bfac0e5a-09ea-47cd-b3f8-2116ef56f1c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:42.996231 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1140/2263] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:42 2022] DELETE /identity/v3/projects/bd98346cd7634c1195be359c27172368 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:42.998582 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0cdd8-8c3d-4a34-83bf-cc766ad219a8 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:42.999676 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0cdd8-8c3d-4a34-83bf-cc766ad219a8 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:43.000123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0cdd8-8c3d-4a34-83bf-cc766ad219a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:43.073995 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-dbd0cdd8-8c3d-4a34-83bf-cc766ad219a8 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:43.075466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-dbd0cdd8-8c3d-4a34-83bf-cc766ad219a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:43.152189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:43.176607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1124/2264] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:43.184927 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ce6b3-4c61-4c75-a1f4-02be1d2d5801 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:43.185301 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ce6b3-4c61-4c75-a1f4-02be1d2d5801 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:43.185529 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ce6b3-4c61-4c75-a1f4-02be1d2d5801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:43.226788 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-061ce6b3-4c61-4c75-a1f4-02be1d2d5801 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:43.232056 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-061ce6b3-4c61-4c75-a1f4-02be1d2d5801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:43.255322 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1141/2265] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:43.259527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0cdd8-8c3d-4a34-83bf-cc766ad219a8 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:43.278223 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0cdd8-8c3d-4a34-83bf-cc766ad219a8 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:43.309944 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06daa410-2bc0-4989-83ea-5bdf814a0597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:43.326401 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06daa410-2bc0-4989-83ea-5bdf814a0597 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:43.328702 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06daa410-2bc0-4989-83ea-5bdf814a0597 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:43.330505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06daa410-2bc0-4989-83ea-5bdf814a0597 admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:43.332983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06daa410-2bc0-4989-83ea-5bdf814a0597 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:46:43.358458 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06daa410-2bc0-4989-83ea-5bdf814a0597 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:43.372244 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06daa410-2bc0-4989-83ea-5bdf814a0597 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:43.424548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1125/2266] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:46:43.434576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ce6b3-4c61-4c75-a1f4-02be1d2d5801 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:43.435399 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:43.443586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ce6b3-4c61-4c75-a1f4-02be1d2d5801 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:43.460204 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ba748-f6f2-4992-97ed-22f8c930924c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:43.460899 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ba748-f6f2-4992-97ed-22f8c930924c admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:43.461184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ba748-f6f2-4992-97ed-22f8c930924c admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:43.461464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ba748-f6f2-4992-97ed-22f8c930924c admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:43.462189 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ba748-f6f2-4992-97ed-22f8c930924c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:43.464421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ba748-f6f2-4992-97ed-22f8c930924c admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:43.537838 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1142/2267] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:43 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:43.538787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06daa410-2bc0-4989-83ea-5bdf814a0597 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:43.557760 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06daa410-2bc0-4989-83ea-5bdf814a0597 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:43.683793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfb238-1826-483d-bcec-219d965ef2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:43.684900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfb238-1826-483d-bcec-219d965ef2eb admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:43.685396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfb238-1826-483d-bcec-219d965ef2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:43.685912 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfb238-1826-483d-bcec-219d965ef2eb admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:43.686966 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfb238-1826-483d-bcec-219d965ef2eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:43.698479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfb238-1826-483d-bcec-219d965ef2eb admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:43.754935 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1126/2268] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:43 2022] POST /identity/v3/users => generated 287 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:46:43.758754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ba748-f6f2-4992-97ed-22f8c930924c admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:43.779294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ba748-f6f2-4992-97ed-22f8c930924c admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:43.785410 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc3b51-1035-4eb3-9e72-6470f6ca47e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:43.786713 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc3b51-1035-4eb3-9e72-6470f6ca47e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:43.787328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc3b51-1035-4eb3-9e72-6470f6ca47e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:43.787991 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc3b51-1035-4eb3-9e72-6470f6ca47e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:43.788970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc3b51-1035-4eb3-9e72-6470f6ca47e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:43.791732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc3b51-1035-4eb3-9e72-6470f6ca47e7 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:43.811336 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1143/2269] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:43.816984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfb238-1826-483d-bcec-219d965ef2eb admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:43.831219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfb238-1826-483d-bcec-219d965ef2eb admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:43.851929 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10ef0b-9613-4ab5-a864-c24687dab7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:43.853205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10ef0b-9613-4ab5-a864-c24687dab7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:43.853821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10ef0b-9613-4ab5-a864-c24687dab7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:43.854108 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10ef0b-9613-4ab5-a864-c24687dab7a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:43.855111 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10ef0b-9613-4ab5-a864-c24687dab7a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:43.857909 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10ef0b-9613-4ab5-a864-c24687dab7a8 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:43.875296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1127/2270] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:43.876982 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc3b51-1035-4eb3-9e72-6470f6ca47e7 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:43.889251 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc3b51-1035-4eb3-9e72-6470f6ca47e7 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:43.914201 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c3722-d6a8-49f3-bebf-a40bacedbcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:43.925337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c3722-d6a8-49f3-bebf-a40bacedbcf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:43.937042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c3722-d6a8-49f3-bebf-a40bacedbcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:43.938670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c3722-d6a8-49f3-bebf-a40bacedbcf1 admin admin] PATH_INFO: `/v3/projects/1710997398f744df9c96831e2ece47b5/users/a759b3053e4e45049136949a84a83510/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:43.972871 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c3722-d6a8-49f3-bebf-a40bacedbcf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1710997398f744df9c96831e2ece47b5, user_id=a759b3053e4e45049136949a84a83510, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:43.975868 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c3722-d6a8-49f3-bebf-a40bacedbcf1 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:43.994872 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1144/2271] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:43 2022] PUT /identity/v3/projects/1710997398f744df9c96831e2ece47b5/users/a759b3053e4e45049136949a84a83510/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:46:44.006401 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2ffa1-8777-4a8f-b664-5a81b83ba1b5 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:44.007266 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2ffa1-8777-4a8f-b664-5a81b83ba1b5 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:44.007266 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2ffa1-8777-4a8f-b664-5a81b83ba1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:44.097799 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-46d2ffa1-8777-4a8f-b664-5a81b83ba1b5 None None] MFA Rules not processed for user `a759b3053e4e45049136949a84a83510`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:44.099156 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-46d2ffa1-8777-4a8f-b664-5a81b83ba1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:44.223639 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1128/2272] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:44.244337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c3722-d6a8-49f3-bebf-a40bacedbcf1 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:44.246780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5b8c3722-d6a8-49f3-bebf-a40bacedbcf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:44.366812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c3722-d6a8-49f3-bebf-a40bacedbcf1 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:44.373212 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a1680-fce1-475d-8064-a0fc09d06f1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:44.373945 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a1680-fce1-475d-8064-a0fc09d06f1d service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:44.374229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a1680-fce1-475d-8064-a0fc09d06f1d service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:44.374514 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a1680-fce1-475d-8064-a0fc09d06f1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:44.385804 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a1680-fce1-475d-8064-a0fc09d06f1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:44.386969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a1680-fce1-475d-8064-a0fc09d06f1d service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:44.401124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1145/2273] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:44.427110 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2ffa1-8777-4a8f-b664-5a81b83ba1b5 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:44.502256 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:44.525672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2ffa1-8777-4a8f-b664-5a81b83ba1b5 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:44.531575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff202d-8de6-407b-8852-650f8e630db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:44.532490 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff202d-8de6-407b-8852-650f8e630db5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:44.532767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff202d-8de6-407b-8852-650f8e630db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:44.533048 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff202d-8de6-407b-8852-650f8e630db5 admin admin] PATH_INFO: `/v3/users/a759b3053e4e45049136949a84a83510` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:44.534453 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff202d-8de6-407b-8852-650f8e630db5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a759b3053e4e45049136949a84a83510)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:44.536549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff202d-8de6-407b-8852-650f8e630db5 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:44.606099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-26ff202d-8de6-407b-8852-650f8e630db5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a759b3053e4e45049136949a84a83510'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:44.625853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-26ff202d-8de6-407b-8852-650f8e630db5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a759b3053e4e45049136949a84a83510'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:44.644479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-26ff202d-8de6-407b-8852-650f8e630db5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a759b3053e4e45049136949a84a83510'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:44.647181 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-26ff202d-8de6-407b-8852-650f8e630db5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a759b3053e4e45049136949a84a83510'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:44.664674 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-26ff202d-8de6-407b-8852-650f8e630db5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a759b3053e4e45049136949a84a83510'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:44.685027 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1129/2274] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:44 2022] DELETE /identity/v3/users/a759b3053e4e45049136949a84a83510 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:44.688567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6136a-9052-46e7-9fe8-34e72da3256f None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:44.689440 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6136a-9052-46e7-9fe8-34e72da3256f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:44.689959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6136a-9052-46e7-9fe8-34e72da3256f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:44.715078 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-6de6136a-9052-46e7-9fe8-34e72da3256f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:44.716436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-6de6136a-9052-46e7-9fe8-34e72da3256f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:44.822481 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1146/2275] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:44.843404 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff202d-8de6-407b-8852-650f8e630db5 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:44.850228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-26ff202d-8de6-407b-8852-650f8e630db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:44.956300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff202d-8de6-407b-8852-650f8e630db5 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:44.960285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31effa02-d954-40a0-a9f9-57953dfcf51c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:44.960940 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31effa02-d954-40a0-a9f9-57953dfcf51c service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:44.961222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31effa02-d954-40a0-a9f9-57953dfcf51c service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:44.961500 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31effa02-d954-40a0-a9f9-57953dfcf51c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:44.977848 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31effa02-d954-40a0-a9f9-57953dfcf51c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:44.978218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31effa02-d954-40a0-a9f9-57953dfcf51c service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:44.994091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1130/2276] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:45.786546 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6136a-9052-46e7-9fe8-34e72da3256f None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:45.788714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-6de6136a-9052-46e7-9fe8-34e72da3256f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:45.809782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6136a-9052-46e7-9fe8-34e72da3256f None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:45.843905 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed12e6-da3b-4a5d-80a6-0909f3ebdee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:45.843905 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed12e6-da3b-4a5d-80a6-0909f3ebdee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:45.843905 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed12e6-da3b-4a5d-80a6-0909f3ebdee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:45.843905 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed12e6-da3b-4a5d-80a6-0909f3ebdee8 admin admin] PATH_INFO: `/v3/projects/1710997398f744df9c96831e2ece47b5` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:45.846333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed12e6-da3b-4a5d-80a6-0909f3ebdee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1710997398f744df9c96831e2ece47b5)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:45.846333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed12e6-da3b-4a5d-80a6-0909f3ebdee8 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:45.864373 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de728817-3828-47ee-9750-7d5e02bdb441 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:45.865060 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de728817-3828-47ee-9750-7d5e02bdb441 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:45.866011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de728817-3828-47ee-9750-7d5e02bdb441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:45.894432 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-de728817-3828-47ee-9750-7d5e02bdb441 None None] MFA Rules not processed for user `72365998f1bb459cbff28ca0b267fe69`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:45.895913 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-de728817-3828-47ee-9750-7d5e02bdb441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:45.918074 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:45.924960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b9ed12e6-da3b-4a5d-80a6-0909f3ebdee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1710997398f744df9c96831e2ece47b5', 'request_id': 'req-b9ed12e6-da3b-4a5d-80a6-0909f3ebdee8'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:45.953894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b9ed12e6-da3b-4a5d-80a6-0909f3ebdee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1710997398f744df9c96831e2ece47b5', 'request_id': 'req-b9ed12e6-da3b-4a5d-80a6-0909f3ebdee8'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:45.971717 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b9ed12e6-da3b-4a5d-80a6-0909f3ebdee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1710997398f744df9c96831e2ece47b5', 'request_id': 'req-b9ed12e6-da3b-4a5d-80a6-0909f3ebdee8'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:45.994064 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b9ed12e6-da3b-4a5d-80a6-0909f3ebdee8 admin admin] The token cache is being invalidate because project 1710997398f744df9c96831e2ece47b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:45.994370 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b9ed12e6-da3b-4a5d-80a6-0909f3ebdee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:45.995409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b9ed12e6-da3b-4a5d-80a6-0909f3ebdee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:45.997899 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1147/2277] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:45 2022] DELETE /identity/v3/projects/1710997398f744df9c96831e2ece47b5 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:46.003482 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1131/2278] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:48.335830 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e2f4b-2385-4792-ae21-aaa4a7112dc7 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:48.336418 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e2f4b-2385-4792-ae21-aaa4a7112dc7 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:48.336418 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e2f4b-2385-4792-ae21-aaa4a7112dc7 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:48.337760 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1148/2279] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jun 22 11:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:46:48.350430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dd853-2046-4741-87a2-3a0ae72b404b None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:48.351185 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dd853-2046-4741-87a2-3a0ae72b404b None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:48.351185 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dd853-2046-4741-87a2-3a0ae72b404b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:48.371424 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-f60dd853-2046-4741-87a2-3a0ae72b404b None None] MFA Rules not processed for user `93630ed5e6dc4a5f958bc7cf1a68a26c`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:48.372769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f60dd853-2046-4741-87a2-3a0ae72b404b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:48.384850 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:48.429369 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e2f4b-2385-4792-ae21-aaa4a7112dc7 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:48.431753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-436e2f4b-2385-4792-ae21-aaa4a7112dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:48.471473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1132/2280] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jun 22 11:46:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:48.483388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60dd853-2046-4741-87a2-3a0ae72b404b None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:48.485522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f60dd853-2046-4741-87a2-3a0ae72b404b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:48.500134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60dd853-2046-4741-87a2-3a0ae72b404b None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:48.506139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a494f33-a374-4bc4-bc16-5d2d761c871d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93630ed5e6dc4a5f958bc7cf1a68a26c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:48.507046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a494f33-a374-4bc4-bc16-5d2d761c871d service placement] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:48.507335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a494f33-a374-4bc4-bc16-5d2d761c871d service placement] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:48.507806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a494f33-a374-4bc4-bc16-5d2d761c871d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:48.516276 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e2f4b-2385-4792-ae21-aaa4a7112dc7 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:48.518490 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a494f33-a374-4bc4-bc16-5d2d761c871d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:48.520371 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a494f33-a374-4bc4-bc16-5d2d761c871d service placement] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:48.521515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f73036-7858-47c3-b1ee-1d86fd2db2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:48.522404 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f73036-7858-47c3-b1ee-1d86fd2db2ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:48.522745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f73036-7858-47c3-b1ee-1d86fd2db2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:48.522972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f73036-7858-47c3-b1ee-1d86fd2db2ad admin admin] PATH_INFO: `/v3/users/3b8a3e35ce254868bf3825cf60139979` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:48.524290 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f73036-7858-47c3-b1ee-1d86fd2db2ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b8a3e35ce254868bf3825cf60139979)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:48.526237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f73036-7858-47c3-b1ee-1d86fd2db2ad admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:48.531985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1133/2281] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:46:48.582876 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c6f73036-7858-47c3-b1ee-1d86fd2db2ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b8a3e35ce254868bf3825cf60139979'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:48.596049 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c6f73036-7858-47c3-b1ee-1d86fd2db2ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b8a3e35ce254868bf3825cf60139979'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:48.604736 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c6f73036-7858-47c3-b1ee-1d86fd2db2ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b8a3e35ce254868bf3825cf60139979'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:48.614265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c6f73036-7858-47c3-b1ee-1d86fd2db2ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b8a3e35ce254868bf3825cf60139979'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:48.614964 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c6f73036-7858-47c3-b1ee-1d86fd2db2ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b8a3e35ce254868bf3825cf60139979'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:48.622704 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1149/2282] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:46:48 2022] DELETE /identity/v3/users/3b8a3e35ce254868bf3825cf60139979 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:48.809761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a494f33-a374-4bc4-bc16-5d2d761c871d service placement] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:48.823653 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a494f33-a374-4bc4-bc16-5d2d761c871d service placement] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:48.888439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d2939-3b57-4535-97ce-67396c386e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:48.889774 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d2939-3b57-4535-97ce-67396c386e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:48.889774 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d2939-3b57-4535-97ce-67396c386e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:48.890138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d2939-3b57-4535-97ce-67396c386e6b admin admin] PATH_INFO: `/v3/projects/35df61dc9f3544a4ad9fa3be324d1e8b` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:48.891141 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d2939-3b57-4535-97ce-67396c386e6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=35df61dc9f3544a4ad9fa3be324d1e8b)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:48.893375 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d2939-3b57-4535-97ce-67396c386e6b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:48.948437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d96d2939-3b57-4535-97ce-67396c386e6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35df61dc9f3544a4ad9fa3be324d1e8b', 'request_id': 'req-d96d2939-3b57-4535-97ce-67396c386e6b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:48.955293 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d96d2939-3b57-4535-97ce-67396c386e6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35df61dc9f3544a4ad9fa3be324d1e8b', 'request_id': 'req-d96d2939-3b57-4535-97ce-67396c386e6b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:48.962117 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d96d2939-3b57-4535-97ce-67396c386e6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35df61dc9f3544a4ad9fa3be324d1e8b', 'request_id': 'req-d96d2939-3b57-4535-97ce-67396c386e6b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:48.973156 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d96d2939-3b57-4535-97ce-67396c386e6b admin admin] The token cache is being invalidate because project 35df61dc9f3544a4ad9fa3be324d1e8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:46:48.973444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d96d2939-3b57-4535-97ce-67396c386e6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:48.974091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d96d2939-3b57-4535-97ce-67396c386e6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:46:48.975706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1134/2283] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:46:48 2022] DELETE /identity/v3/projects/35df61dc9f3544a4ad9fa3be324d1e8b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:48.981130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42246f-da5d-4e3c-8179-56c3d059e2fd None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:48.981421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42246f-da5d-4e3c-8179-56c3d059e2fd None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:48.981700 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42246f-da5d-4e3c-8179-56c3d059e2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:49.000285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-2a42246f-da5d-4e3c-8179-56c3d059e2fd None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:49.001517 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-2a42246f-da5d-4e3c-8179-56c3d059e2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:49.052536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:49.066636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1150/2284] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:46:49.075628 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c83ec-036a-492a-bb29-477df3283dca None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:49.075946 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c83ec-036a-492a-bb29-477df3283dca None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:49.076279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c83ec-036a-492a-bb29-477df3283dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:49.096267 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-a94c83ec-036a-492a-bb29-477df3283dca None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:49.097507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a94c83ec-036a-492a-bb29-477df3283dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:49.105705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1135/2285] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:49.110781 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42246f-da5d-4e3c-8179-56c3d059e2fd None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:49.120704 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42246f-da5d-4e3c-8179-56c3d059e2fd None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:49.125455 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96700a1f-ae36-4388-aa53-47d5ac420690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:49.126108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96700a1f-ae36-4388-aa53-47d5ac420690 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:49.126395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96700a1f-ae36-4388-aa53-47d5ac420690 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:49.126674 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96700a1f-ae36-4388-aa53-47d5ac420690 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:49.127433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96700a1f-ae36-4388-aa53-47d5ac420690 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:46:49.127747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96700a1f-ae36-4388-aa53-47d5ac420690 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:49.129355 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96700a1f-ae36-4388-aa53-47d5ac420690 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:49.143011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1151/2286] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:46:49.147925 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c83ec-036a-492a-bb29-477df3283dca None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:49.160240 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c83ec-036a-492a-bb29-477df3283dca None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:49.165253 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d940d-58f9-47c6-ba2f-e08846fd64f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:49.165912 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d940d-58f9-47c6-ba2f-e08846fd64f0 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:49.166198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d940d-58f9-47c6-ba2f-e08846fd64f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:49.166497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d940d-58f9-47c6-ba2f-e08846fd64f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:49.167182 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d940d-58f9-47c6-ba2f-e08846fd64f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:49.169119 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d940d-58f9-47c6-ba2f-e08846fd64f0 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:49.189703 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:49.205538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1136/2287] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:46:49 2022] POST /identity/v3/projects => generated 389 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:49.206681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96700a1f-ae36-4388-aa53-47d5ac420690 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:49.220875 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96700a1f-ae36-4388-aa53-47d5ac420690 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:49.282630 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f920e0-f356-4d85-9943-9c00bea4786e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:49.283612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f920e0-f356-4d85-9943-9c00bea4786e admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:49.283833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f920e0-f356-4d85-9943-9c00bea4786e admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:49.284116 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f920e0-f356-4d85-9943-9c00bea4786e admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:49.287826 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f920e0-f356-4d85-9943-9c00bea4786e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:49.290275 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f920e0-f356-4d85-9943-9c00bea4786e admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:49.320294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1152/2288] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:46:49 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:46:49.322041 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d940d-58f9-47c6-ba2f-e08846fd64f0 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:49.332194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d940d-58f9-47c6-ba2f-e08846fd64f0 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:49.336971 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df422df-ef67-43d8-b349-500394d739d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:49.337695 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df422df-ef67-43d8-b349-500394d739d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:49.338069 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df422df-ef67-43d8-b349-500394d739d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:49.338411 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df422df-ef67-43d8-b349-500394d739d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:49.339203 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df422df-ef67-43d8-b349-500394d739d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:49.340882 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df422df-ef67-43d8-b349-500394d739d7 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:49.355311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1137/2289] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:49.356934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f920e0-f356-4d85-9943-9c00bea4786e admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:49.368908 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f920e0-f356-4d85-9943-9c00bea4786e admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:49.373949 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c534867-0c8f-4a13-a36a-c5b9bb86e463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:49.375221 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c534867-0c8f-4a13-a36a-c5b9bb86e463 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:49.375568 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c534867-0c8f-4a13-a36a-c5b9bb86e463 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:49.375843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c534867-0c8f-4a13-a36a-c5b9bb86e463 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:49.376570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c534867-0c8f-4a13-a36a-c5b9bb86e463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:49.378378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c534867-0c8f-4a13-a36a-c5b9bb86e463 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:49.390007 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1153/2290] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:46:49.394413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df422df-ef67-43d8-b349-500394d739d7 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:49.408423 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df422df-ef67-43d8-b349-500394d739d7 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:49.413177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091f5bc-dae4-4f60-a788-daa7428a6c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:49.414096 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091f5bc-dae4-4f60-a788-daa7428a6c50 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:49.414381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091f5bc-dae4-4f60-a788-daa7428a6c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:49.414654 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091f5bc-dae4-4f60-a788-daa7428a6c50 admin admin] PATH_INFO: `/v3/projects/6f64ad44d3c247caa7d106861916f245/users/034858240ac34e549aa5e0754726f9c6/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:49.431784 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091f5bc-dae4-4f60-a788-daa7428a6c50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f64ad44d3c247caa7d106861916f245, user_id=034858240ac34e549aa5e0754726f9c6, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:49.434424 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091f5bc-dae4-4f60-a788-daa7428a6c50 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:49.448306 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1138/2291] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:46:49 2022] PUT /identity/v3/projects/6f64ad44d3c247caa7d106861916f245/users/034858240ac34e549aa5e0754726f9c6/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:46:49.453966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72683df-4835-49bb-a254-7b0d0452395f None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:49.454286 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72683df-4835-49bb-a254-7b0d0452395f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:49.454681 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72683df-4835-49bb-a254-7b0d0452395f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:49.477444 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-b72683df-4835-49bb-a254-7b0d0452395f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:49.478786 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b72683df-4835-49bb-a254-7b0d0452395f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:49.549640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1154/2292] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:49.560717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091f5bc-dae4-4f60-a788-daa7428a6c50 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:49.562668 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b091f5bc-dae4-4f60-a788-daa7428a6c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:49.634607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091f5bc-dae4-4f60-a788-daa7428a6c50 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:49.638485 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13216be-1fcb-48a0-9004-91f1f82bdedc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:49.639152 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13216be-1fcb-48a0-9004-91f1f82bdedc service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:49.639462 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13216be-1fcb-48a0-9004-91f1f82bdedc service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:49.639741 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13216be-1fcb-48a0-9004-91f1f82bdedc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:49.651154 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13216be-1fcb-48a0-9004-91f1f82bdedc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:49.652734 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13216be-1fcb-48a0-9004-91f1f82bdedc service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:49.667224 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1139/2293] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:46:54.645778 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03748204-b4d2-446e-8b15-a4f1f89f46f6 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:54.646450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03748204-b4d2-446e-8b15-a4f1f89f46f6 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:54.646450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03748204-b4d2-446e-8b15-a4f1f89f46f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:54.684358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-03748204-b4d2-446e-8b15-a4f1f89f46f6 None None] MFA Rules not processed for user `034858240ac34e549aa5e0754726f9c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:54.685607 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-03748204-b4d2-446e-8b15-a4f1f89f46f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:54.786081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:46:54.816082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1155/2294] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:46:54.833011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13216be-1fcb-48a0-9004-91f1f82bdedc service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:46:54.835155 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d13216be-1fcb-48a0-9004-91f1f82bdedc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:54.939184 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13216be-1fcb-48a0-9004-91f1f82bdedc service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:46:54.945280 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a6b1e-2e74-428b-8ec6-f8840505c513 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:46:54.945957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a6b1e-2e74-428b-8ec6-f8840505c513 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:54.946797 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a6b1e-2e74-428b-8ec6-f8840505c513 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:54.946797 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a6b1e-2e74-428b-8ec6-f8840505c513 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:54.960788 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a6b1e-2e74-428b-8ec6-f8840505c513 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:46:54.965368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a6b1e-2e74-428b-8ec6-f8840505c513 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:46:54.981549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1140/2295] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:46:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:46:56.647493 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659fffd-9694-4773-b776-e28ff6c4bee3 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:46:56.648015 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659fffd-9694-4773-b776-e28ff6c4bee3 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:46:56.648106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659fffd-9694-4773-b776-e28ff6c4bee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:46:56.668357 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f659fffd-9694-4773-b776-e28ff6c4bee3 None None] MFA Rules not processed for user `034858240ac34e549aa5e0754726f9c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:46:56.669834 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f659fffd-9694-4773-b776-e28ff6c4bee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:46:56.679186 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1156/2296] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:46:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:04.971764 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=75,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:06.661513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=8 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:15.719414 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a6b1e-2e74-428b-8ec6-f8840505c513 service nova] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:15.721063 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c37a6b1e-2e74-428b-8ec6-f8840505c513 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:15.735679 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a6b1e-2e74-428b-8ec6-f8840505c513 service nova] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:15.740705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e088d-310e-4718-bf9c-dab26110c0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:15.741592 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e088d-310e-4718-bf9c-dab26110c0c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:15.741872 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e088d-310e-4718-bf9c-dab26110c0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:15.742176 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e088d-310e-4718-bf9c-dab26110c0c9 admin admin] PATH_INFO: `/v3/users/7fcf899d893b4122878db3365f9ef120` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:15.743845 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e088d-310e-4718-bf9c-dab26110c0c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fcf899d893b4122878db3365f9ef120)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:15.745355 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e088d-310e-4718-bf9c-dab26110c0c9 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:15.795567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-e66e088d-310e-4718-bf9c-dab26110c0c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fcf899d893b4122878db3365f9ef120'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:15.809524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-e66e088d-310e-4718-bf9c-dab26110c0c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fcf899d893b4122878db3365f9ef120'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:15.818491 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-e66e088d-310e-4718-bf9c-dab26110c0c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fcf899d893b4122878db3365f9ef120'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:15.819194 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-e66e088d-310e-4718-bf9c-dab26110c0c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fcf899d893b4122878db3365f9ef120'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:15.828465 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-e66e088d-310e-4718-bf9c-dab26110c0c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fcf899d893b4122878db3365f9ef120'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:15.836385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1141/2297] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:47:15 2022] DELETE /identity/v3/users/7fcf899d893b4122878db3365f9ef120 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:47:15.841090 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bc33d-69b5-4234-b48c-7bee3e165adf None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:15.841541 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bc33d-69b5-4234-b48c-7bee3e165adf None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:15.841653 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bc33d-69b5-4234-b48c-7bee3e165adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:15.858737 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-462bc33d-69b5-4234-b48c-7bee3e165adf None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:15.859961 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-462bc33d-69b5-4234-b48c-7bee3e165adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:15.919606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1157/2298] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:15.930032 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e088d-310e-4718-bf9c-dab26110c0c9 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:15.931476 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e66e088d-310e-4718-bf9c-dab26110c0c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:16.002684 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e088d-310e-4718-bf9c-dab26110c0c9 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:16.006628 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ad333f-ec99-445e-9284-01e68e4068c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:16.007278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad333f-ec99-445e-9284-01e68e4068c1 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:16.007591 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad333f-ec99-445e-9284-01e68e4068c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:16.007874 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad333f-ec99-445e-9284-01e68e4068c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:16.019511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad333f-ec99-445e-9284-01e68e4068c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:16.020933 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ad333f-ec99-445e-9284-01e68e4068c1 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:16.032027 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1142/2299] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:47:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:47:16.581870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bc33d-69b5-4234-b48c-7bee3e165adf None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:16.583887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-462bc33d-69b5-4234-b48c-7bee3e165adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:16.601033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bc33d-69b5-4234-b48c-7bee3e165adf None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:16.608271 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2f198-7576-4631-afd8-a7655723aef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:16.609350 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2f198-7576-4631-afd8-a7655723aef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:16.609942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2f198-7576-4631-afd8-a7655723aef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:16.610368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2f198-7576-4631-afd8-a7655723aef0 admin admin] PATH_INFO: `/v3/projects/7ea4f3275fab42338617fe09eafdd143` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:16.612617 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2f198-7576-4631-afd8-a7655723aef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ea4f3275fab42338617fe09eafdd143)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:16.625098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2f198-7576-4631-afd8-a7655723aef0 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:16.677145 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-27c2f198-7576-4631-afd8-a7655723aef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ea4f3275fab42338617fe09eafdd143', 'request_id': 'req-27c2f198-7576-4631-afd8-a7655723aef0'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:16.683160 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-27c2f198-7576-4631-afd8-a7655723aef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ea4f3275fab42338617fe09eafdd143', 'request_id': 'req-27c2f198-7576-4631-afd8-a7655723aef0'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:16.690552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-27c2f198-7576-4631-afd8-a7655723aef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ea4f3275fab42338617fe09eafdd143', 'request_id': 'req-27c2f198-7576-4631-afd8-a7655723aef0'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:16.702389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-27c2f198-7576-4631-afd8-a7655723aef0 admin admin] The token cache is being invalidate because project 7ea4f3275fab42338617fe09eafdd143 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:47:16.702674 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-27c2f198-7576-4631-afd8-a7655723aef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:16.703305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-27c2f198-7576-4631-afd8-a7655723aef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:16.704908 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1158/2300] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:47:16 2022] DELETE /identity/v3/projects/7ea4f3275fab42338617fe09eafdd143 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:47:16.709694 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee4999-f42c-4c8a-99c2-cc1d11180d77 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:16.709983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee4999-f42c-4c8a-99c2-cc1d11180d77 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:16.710261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee4999-f42c-4c8a-99c2-cc1d11180d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:16.728328 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-08ee4999-f42c-4c8a-99c2-cc1d11180d77 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:16.729512 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-08ee4999-f42c-4c8a-99c2-cc1d11180d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:16.789453 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1143/2301] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:16.798427 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e40e54-a914-475a-9b4b-28db5227c414 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:16.798720 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e40e54-a914-475a-9b4b-28db5227c414 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:16.799009 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e40e54-a914-475a-9b4b-28db5227c414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:16.818905 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-52e40e54-a914-475a-9b4b-28db5227c414 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:16.820134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-52e40e54-a914-475a-9b4b-28db5227c414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:16.828515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1159/2302] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:16.832720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee4999-f42c-4c8a-99c2-cc1d11180d77 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:16.843048 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee4999-f42c-4c8a-99c2-cc1d11180d77 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:16.847755 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f55087-ba80-4bd5-abb3-21a2a0a0c4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:16.848403 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f55087-ba80-4bd5-abb3-21a2a0a0c4ac admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:16.848725 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f55087-ba80-4bd5-abb3-21a2a0a0c4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:16.849005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f55087-ba80-4bd5-abb3-21a2a0a0c4ac admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:16.849765 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f55087-ba80-4bd5-abb3-21a2a0a0c4ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:47:16.850072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f55087-ba80-4bd5-abb3-21a2a0a0c4ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:16.851721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f55087-ba80-4bd5-abb3-21a2a0a0c4ac admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:16.863917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1144/2303] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:47:16.868296 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e40e54-a914-475a-9b4b-28db5227c414 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:16.877573 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e40e54-a914-475a-9b4b-28db5227c414 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:16.882528 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad8f64-f486-4b9c-aaf3-6f19b590cee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:16.883188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad8f64-f486-4b9c-aaf3-6f19b590cee8 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:16.883516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad8f64-f486-4b9c-aaf3-6f19b590cee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:16.883799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad8f64-f486-4b9c-aaf3-6f19b590cee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:16.884492 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad8f64-f486-4b9c-aaf3-6f19b590cee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:16.886438 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad8f64-f486-4b9c-aaf3-6f19b590cee8 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:16.913019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1160/2304] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:47:16 2022] POST /identity/v3/projects => generated 359 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:47:16.917162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f55087-ba80-4bd5-abb3-21a2a0a0c4ac admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:16.922391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:16.929188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f55087-ba80-4bd5-abb3-21a2a0a0c4ac admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:16.987614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ee710-8c42-4587-971f-9b3b253e60a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:16.988288 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ee710-8c42-4587-971f-9b3b253e60a2 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:16.988568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ee710-8c42-4587-971f-9b3b253e60a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:16.988922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ee710-8c42-4587-971f-9b3b253e60a2 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:16.989588 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ee710-8c42-4587-971f-9b3b253e60a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:16.991731 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ee710-8c42-4587-971f-9b3b253e60a2 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:17.011860 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1145/2305] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:47:16 2022] POST /identity/v3/users => generated 280 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:47:17.017213 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad8f64-f486-4b9c-aaf3-6f19b590cee8 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:17.027547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad8f64-f486-4b9c-aaf3-6f19b590cee8 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:17.032767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10961eec-1640-4e05-9564-5024fb16c607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:17.033420 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10961eec-1640-4e05-9564-5024fb16c607 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:17.033702 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10961eec-1640-4e05-9564-5024fb16c607 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:17.033977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10961eec-1640-4e05-9564-5024fb16c607 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:17.034695 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10961eec-1640-4e05-9564-5024fb16c607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:17.036322 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10961eec-1640-4e05-9564-5024fb16c607 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:17.048128 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1161/2306] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:47:17.052329 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ee710-8c42-4587-971f-9b3b253e60a2 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:17.062683 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ee710-8c42-4587-971f-9b3b253e60a2 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:17.067417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816b760-b103-4a0e-ac6c-892c92935c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:17.068079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816b760-b103-4a0e-ac6c-892c92935c35 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:17.068365 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816b760-b103-4a0e-ac6c-892c92935c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:17.068637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816b760-b103-4a0e-ac6c-892c92935c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:17.069360 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816b760-b103-4a0e-ac6c-892c92935c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:17.070941 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816b760-b103-4a0e-ac6c-892c92935c35 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:17.081801 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1146/2307] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:47:17.085887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10961eec-1640-4e05-9564-5024fb16c607 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:17.096527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10961eec-1640-4e05-9564-5024fb16c607 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:17.101110 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c77d98-2a8c-4a5f-acfc-39b04f394811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:17.101972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c77d98-2a8c-4a5f-acfc-39b04f394811 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:17.102251 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c77d98-2a8c-4a5f-acfc-39b04f394811 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:17.102524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c77d98-2a8c-4a5f-acfc-39b04f394811 admin admin] PATH_INFO: `/v3/projects/28173c350c47425bb1a69af626c4812c/users/ff584a7bd22b4e1f987f496e2a0b7a57/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:17.116771 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c77d98-2a8c-4a5f-acfc-39b04f394811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28173c350c47425bb1a69af626c4812c, user_id=ff584a7bd22b4e1f987f496e2a0b7a57, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:17.119444 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c77d98-2a8c-4a5f-acfc-39b04f394811 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:17.131984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1162/2308] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:47:17 2022] PUT /identity/v3/projects/28173c350c47425bb1a69af626c4812c/users/ff584a7bd22b4e1f987f496e2a0b7a57/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:47:17.143412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511065f-84c3-4ea3-b063-a251f3df7321 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:17.143811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511065f-84c3-4ea3-b063-a251f3df7321 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:17.144159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511065f-84c3-4ea3-b063-a251f3df7321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:17.171231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-e511065f-84c3-4ea3-b063-a251f3df7321 None None] MFA Rules not processed for user `ff584a7bd22b4e1f987f496e2a0b7a57`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:17.172487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e511065f-84c3-4ea3-b063-a251f3df7321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:17.249814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1147/2309] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:17 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:17.294636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c77d98-2a8c-4a5f-acfc-39b04f394811 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:17.297271 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f9c77d98-2a8c-4a5f-acfc-39b04f394811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:17.320511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:17.363603 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c77d98-2a8c-4a5f-acfc-39b04f394811 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:17.367471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a98479-4acc-49cb-8fda-6c14c83ccb6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:17.368133 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a98479-4acc-49cb-8fda-6c14c83ccb6b service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:17.368423 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a98479-4acc-49cb-8fda-6c14c83ccb6b service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:17.368707 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a98479-4acc-49cb-8fda-6c14c83ccb6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:17.377615 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a98479-4acc-49cb-8fda-6c14c83ccb6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:17.379096 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a98479-4acc-49cb-8fda-6c14c83ccb6b service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:17.388258 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1163/2310] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:47:17 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:47:17.402922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511065f-84c3-4ea3-b063-a251f3df7321 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:17.463672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511065f-84c3-4ea3-b063-a251f3df7321 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:17.468881 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa456c0-c022-4b11-88f4-2142269bdd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:17.469733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa456c0-c022-4b11-88f4-2142269bdd2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:17.470011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa456c0-c022-4b11-88f4-2142269bdd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:17.470281 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa456c0-c022-4b11-88f4-2142269bdd2d admin admin] PATH_INFO: `/v3/users/ff584a7bd22b4e1f987f496e2a0b7a57` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:17.471493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa456c0-c022-4b11-88f4-2142269bdd2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff584a7bd22b4e1f987f496e2a0b7a57)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:17.473407 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa456c0-c022-4b11-88f4-2142269bdd2d admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:17.486295 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:17.519763 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4aa456c0-c022-4b11-88f4-2142269bdd2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff584a7bd22b4e1f987f496e2a0b7a57'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:17.534212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4aa456c0-c022-4b11-88f4-2142269bdd2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff584a7bd22b4e1f987f496e2a0b7a57'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:17.542300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4aa456c0-c022-4b11-88f4-2142269bdd2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff584a7bd22b4e1f987f496e2a0b7a57'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:17.542960 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4aa456c0-c022-4b11-88f4-2142269bdd2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff584a7bd22b4e1f987f496e2a0b7a57'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:17.550646 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4aa456c0-c022-4b11-88f4-2142269bdd2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff584a7bd22b4e1f987f496e2a0b7a57'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:17.556773 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1148/2311] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:47:17 2022] DELETE /identity/v3/users/ff584a7bd22b4e1f987f496e2a0b7a57 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:47:17.561063 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a6783-7070-4393-b074-fd4594e6ac51 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:17.561459 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a6783-7070-4393-b074-fd4594e6ac51 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:17.561733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a6783-7070-4393-b074-fd4594e6ac51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:17.579049 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-ca0a6783-7070-4393-b074-fd4594e6ac51 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:17.580342 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ca0a6783-7070-4393-b074-fd4594e6ac51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:17.645782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1164/2312] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:47:17.664323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa456c0-c022-4b11-88f4-2142269bdd2d admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:17.665723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-4aa456c0-c022-4b11-88f4-2142269bdd2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:17.751220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa456c0-c022-4b11-88f4-2142269bdd2d admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:17.755063 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1523745-77f3-4dc1-8dd1-a34eacd38813 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:17.755612 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1523745-77f3-4dc1-8dd1-a34eacd38813 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:17.755895 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1523745-77f3-4dc1-8dd1-a34eacd38813 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:17.756171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1523745-77f3-4dc1-8dd1-a34eacd38813 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:17.768636 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1523745-77f3-4dc1-8dd1-a34eacd38813 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:17.770311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1523745-77f3-4dc1-8dd1-a34eacd38813 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:17.782108 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1149/2313] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:47:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:47:18.100157 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a6783-7070-4393-b074-fd4594e6ac51 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:18.101698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ca0a6783-7070-4393-b074-fd4594e6ac51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:18.115227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a6783-7070-4393-b074-fd4594e6ac51 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:18.120368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee490cb-8e8b-408b-89a8-f41538ef4ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:18.121263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee490cb-8e8b-408b-89a8-f41538ef4ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:18.121546 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee490cb-8e8b-408b-89a8-f41538ef4ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:18.121828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee490cb-8e8b-408b-89a8-f41538ef4ef6 admin admin] PATH_INFO: `/v3/projects/28173c350c47425bb1a69af626c4812c` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:18.123006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee490cb-8e8b-408b-89a8-f41538ef4ef6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28173c350c47425bb1a69af626c4812c)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:18.126786 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee490cb-8e8b-408b-89a8-f41538ef4ef6 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:18.177792 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-5ee490cb-8e8b-408b-89a8-f41538ef4ef6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28173c350c47425bb1a69af626c4812c', 'request_id': 'req-5ee490cb-8e8b-408b-89a8-f41538ef4ef6'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:18.184106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-5ee490cb-8e8b-408b-89a8-f41538ef4ef6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28173c350c47425bb1a69af626c4812c', 'request_id': 'req-5ee490cb-8e8b-408b-89a8-f41538ef4ef6'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:18.192312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-5ee490cb-8e8b-408b-89a8-f41538ef4ef6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28173c350c47425bb1a69af626c4812c', 'request_id': 'req-5ee490cb-8e8b-408b-89a8-f41538ef4ef6'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:18.204540 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-5ee490cb-8e8b-408b-89a8-f41538ef4ef6 admin admin] The token cache is being invalidate because project 28173c350c47425bb1a69af626c4812c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:47:18.204827 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-5ee490cb-8e8b-408b-89a8-f41538ef4ef6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:18.205459 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-5ee490cb-8e8b-408b-89a8-f41538ef4ef6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:18.207008 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1165/2314] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:47:18 2022] DELETE /identity/v3/projects/28173c350c47425bb1a69af626c4812c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:47:18.211896 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f8701-7785-4b3e-b14e-da06bc13a32f None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:18.212189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f8701-7785-4b3e-b14e-da06bc13a32f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:18.212472 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f8701-7785-4b3e-b14e-da06bc13a32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:18.230320 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c03f8701-7785-4b3e-b14e-da06bc13a32f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:18.231547 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c03f8701-7785-4b3e-b14e-da06bc13a32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:18.290384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1150/2315] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:18.298094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42c592-e577-4682-83f7-2b4d6694b699 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:18.298383 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42c592-e577-4682-83f7-2b4d6694b699 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:18.298659 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42c592-e577-4682-83f7-2b4d6694b699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:18.317131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-9a42c592-e577-4682-83f7-2b4d6694b699 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:18.318329 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-9a42c592-e577-4682-83f7-2b4d6694b699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:18.326163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1166/2316] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:18.331114 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f8701-7785-4b3e-b14e-da06bc13a32f None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:18.341041 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f8701-7785-4b3e-b14e-da06bc13a32f None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:18.345791 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbef0e-b7fc-4a7a-89d0-b0e593293c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:18.346437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbef0e-b7fc-4a7a-89d0-b0e593293c35 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:18.346727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbef0e-b7fc-4a7a-89d0-b0e593293c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:18.347006 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbef0e-b7fc-4a7a-89d0-b0e593293c35 admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:18.347770 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbef0e-b7fc-4a7a-89d0-b0e593293c35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:47:18.348079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbef0e-b7fc-4a7a-89d0-b0e593293c35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:18.349664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbef0e-b7fc-4a7a-89d0-b0e593293c35 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:18.361917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1151/2317] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:47:18.366090 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42c592-e577-4682-83f7-2b4d6694b699 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:18.376210 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42c592-e577-4682-83f7-2b4d6694b699 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:18.381518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad67ba-aff6-47b5-89b7-3a7a2bfe6c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:18.382358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad67ba-aff6-47b5-89b7-3a7a2bfe6c1d admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:18.382639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad67ba-aff6-47b5-89b7-3a7a2bfe6c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:18.382914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad67ba-aff6-47b5-89b7-3a7a2bfe6c1d admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:18.383685 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad67ba-aff6-47b5-89b7-3a7a2bfe6c1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:18.385596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad67ba-aff6-47b5-89b7-3a7a2bfe6c1d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:18.417194 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1167/2318] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:47:18 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:47:18.421711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbef0e-b7fc-4a7a-89d0-b0e593293c35 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:18.432187 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbef0e-b7fc-4a7a-89d0-b0e593293c35 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:18.452656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:18.496256 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027dbc53-b61a-4eac-8b46-12bfc60b0411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:18.497568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027dbc53-b61a-4eac-8b46-12bfc60b0411 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:18.497568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027dbc53-b61a-4eac-8b46-12bfc60b0411 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:18.497568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027dbc53-b61a-4eac-8b46-12bfc60b0411 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:18.498334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027dbc53-b61a-4eac-8b46-12bfc60b0411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:18.500437 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027dbc53-b61a-4eac-8b46-12bfc60b0411 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:18.526532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1152/2319] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:47:18 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:47:18.527855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad67ba-aff6-47b5-89b7-3a7a2bfe6c1d admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:18.539047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad67ba-aff6-47b5-89b7-3a7a2bfe6c1d admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:18.544813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ce698-70b0-4638-93d8-03886254614a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:18.545518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ce698-70b0-4638-93d8-03886254614a admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:18.545801 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ce698-70b0-4638-93d8-03886254614a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:18.546076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ce698-70b0-4638-93d8-03886254614a admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:18.546807 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ce698-70b0-4638-93d8-03886254614a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:18.548452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ce698-70b0-4638-93d8-03886254614a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:18.559508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1168/2320] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:47:18.564430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027dbc53-b61a-4eac-8b46-12bfc60b0411 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:18.574730 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027dbc53-b61a-4eac-8b46-12bfc60b0411 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:18.580749 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114ceec-9332-459a-9de0-90192cbabaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:18.581494 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114ceec-9332-459a-9de0-90192cbabaa2 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:18.581729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114ceec-9332-459a-9de0-90192cbabaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:18.582013 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114ceec-9332-459a-9de0-90192cbabaa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:18.582725 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6114ceec-9332-459a-9de0-90192cbabaa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:18.584730 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6114ceec-9332-459a-9de0-90192cbabaa2 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:18.599231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1153/2321] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:47:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:47:18.601065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ce698-70b0-4638-93d8-03886254614a admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:18.611171 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ce698-70b0-4638-93d8-03886254614a admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:18.616951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5689071-dce9-46ec-a177-1addc20507d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:18.617849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5689071-dce9-46ec-a177-1addc20507d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:18.618132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5689071-dce9-46ec-a177-1addc20507d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:18.618408 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5689071-dce9-46ec-a177-1addc20507d5 admin admin] PATH_INFO: `/v3/projects/3c78f985b56446978713e0645381509e/users/34c9ae5b04924677a19d1e3bc6247d47/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:18.621230 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:18.637472 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5689071-dce9-46ec-a177-1addc20507d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c78f985b56446978713e0645381509e, user_id=34c9ae5b04924677a19d1e3bc6247d47, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:18.640786 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5689071-dce9-46ec-a177-1addc20507d5 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:18.654197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1169/2322] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:47:18 2022] PUT /identity/v3/projects/3c78f985b56446978713e0645381509e/users/34c9ae5b04924677a19d1e3bc6247d47/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:47:18.665895 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60768b92-edec-4393-94a2-033b868f1098 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:18.666187 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60768b92-edec-4393-94a2-033b868f1098 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:18.666468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60768b92-edec-4393-94a2-033b868f1098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:18.692815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-60768b92-edec-4393-94a2-033b868f1098 None None] MFA Rules not processed for user `34c9ae5b04924677a19d1e3bc6247d47`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:18.694030 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-60768b92-edec-4393-94a2-033b868f1098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:18.761595 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1154/2323] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:18.774156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5689071-dce9-46ec-a177-1addc20507d5 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:18.775951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e5689071-dce9-46ec-a177-1addc20507d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:18.834870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5689071-dce9-46ec-a177-1addc20507d5 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:18.838836 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742173f-3484-4acd-9c44-d0a8232bc678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:18.839545 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742173f-3484-4acd-9c44-d0a8232bc678 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:18.839810 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742173f-3484-4acd-9c44-d0a8232bc678 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:18.840108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a742173f-3484-4acd-9c44-d0a8232bc678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:18.849177 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a742173f-3484-4acd-9c44-d0a8232bc678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:18.850658 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a742173f-3484-4acd-9c44-d0a8232bc678 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:18.861936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1170/2324] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:47:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:47:24.409975 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bdaff-b5cb-4cfd-8dfd-c680e0810bc3 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:24.410423 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bdaff-b5cb-4cfd-8dfd-c680e0810bc3 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:24.410530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bdaff-b5cb-4cfd-8dfd-c680e0810bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:24.428117 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-e26bdaff-b5cb-4cfd-8dfd-c680e0810bc3 None None] MFA Rules not processed for user `955c44ffe1bd426e900c7405df44d1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:24.429304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e26bdaff-b5cb-4cfd-8dfd-c680e0810bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:24.486517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1155/2325] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jun 22 11:47:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:24.542133 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742173f-3484-4acd-9c44-d0a8232bc678 service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:24.543634 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-a742173f-3484-4acd-9c44-d0a8232bc678 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:24.625695 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a742173f-3484-4acd-9c44-d0a8232bc678 service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:24.631003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6af5b1-67bb-4382-835f-662665f1fec8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd642e290e44e67b56eb3b8361d7af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:24.631697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6af5b1-67bb-4382-835f-662665f1fec8 service swift] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:24.631985 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6af5b1-67bb-4382-835f-662665f1fec8 service swift] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:24.632269 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6af5b1-67bb-4382-835f-662665f1fec8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:24.641114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6af5b1-67bb-4382-835f-662665f1fec8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:24.642972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6af5b1-67bb-4382-835f-662665f1fec8 service swift] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:24.651116 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1171/2326] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jun 22 11:47:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 22 11:47:25.258758 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd86c1-0b18-4c92-b18d-14e788bfb262 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:25.259140 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd86c1-0b18-4c92-b18d-14e788bfb262 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:25.259490 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd86c1-0b18-4c92-b18d-14e788bfb262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:25.281215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-cdbd86c1-0b18-4c92-b18d-14e788bfb262 None None] MFA Rules not processed for user `955c44ffe1bd426e900c7405df44d1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:25.281215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-cdbd86c1-0b18-4c92-b18d-14e788bfb262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:25.288275 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1156/2327] 10.222.0.36 () {58 vars in 967 bytes} [Wed Jun 22 11:47:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:47:25.301495 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6af5b1-67bb-4382-835f-662665f1fec8 service swift] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:25.312496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6af5b1-67bb-4382-835f-662665f1fec8 service swift] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:25.317504 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c7c8b-e54b-4954-8806-7fddd5594169 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd642e290e44e67b56eb3b8361d7af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:25.318132 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c7c8b-e54b-4954-8806-7fddd5594169 service swift] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:25.318415 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c7c8b-e54b-4954-8806-7fddd5594169 service swift] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:25.318715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c7c8b-e54b-4954-8806-7fddd5594169 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:25.327272 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c7c8b-e54b-4954-8806-7fddd5594169 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:25.328879 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c7c8b-e54b-4954-8806-7fddd5594169 service swift] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:25.336845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1172/2328] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jun 22 11:47:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 22 11:47:25.540681 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ead46-945f-4180-9586-cc11bc890d7e None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:25.540681 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ead46-945f-4180-9586-cc11bc890d7e None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:25.541265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ead46-945f-4180-9586-cc11bc890d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:25.559061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-022ead46-945f-4180-9586-cc11bc890d7e None None] MFA Rules not processed for user `955c44ffe1bd426e900c7405df44d1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:25.560285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-022ead46-945f-4180-9586-cc11bc890d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:25.568040 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1157/2329] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jun 22 11:47:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:25.580339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c7c8b-e54b-4954-8806-7fddd5594169 service swift] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:25.590984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c7c8b-e54b-4954-8806-7fddd5594169 service swift] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:25.596254 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58fc07-3e76-4f3c-a39a-c46850503ec8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd642e290e44e67b56eb3b8361d7af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:25.596916 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58fc07-3e76-4f3c-a39a-c46850503ec8 service swift] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:25.597193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58fc07-3e76-4f3c-a39a-c46850503ec8 service swift] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:25.597473 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58fc07-3e76-4f3c-a39a-c46850503ec8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:25.605744 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a58fc07-3e76-4f3c-a39a-c46850503ec8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:25.607326 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a58fc07-3e76-4f3c-a39a-c46850503ec8 service swift] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:25.615973 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1173/2330] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jun 22 11:47:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 22 11:47:25.998983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8441762-8a6b-4352-ab3e-388113dd0fc4 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:25.999342 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8441762-8a6b-4352-ab3e-388113dd0fc4 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:25.999748 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8441762-8a6b-4352-ab3e-388113dd0fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:26.018079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-f8441762-8a6b-4352-ab3e-388113dd0fc4 None None] MFA Rules not processed for user `955c44ffe1bd426e900c7405df44d1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:26.019282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f8441762-8a6b-4352-ab3e-388113dd0fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:26.027050 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1158/2331] 10.222.0.36 () {58 vars in 967 bytes} [Wed Jun 22 11:47:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:26.039649 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58fc07-3e76-4f3c-a39a-c46850503ec8 service swift] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:26.050225 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a58fc07-3e76-4f3c-a39a-c46850503ec8 service swift] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:26.056397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6b1e2-c8a7-492f-9cdd-3eab5b8ac473 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd642e290e44e67b56eb3b8361d7af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:26.057053 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6b1e2-c8a7-492f-9cdd-3eab5b8ac473 service swift] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:26.057340 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6b1e2-c8a7-492f-9cdd-3eab5b8ac473 service swift] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:26.057622 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6b1e2-c8a7-492f-9cdd-3eab5b8ac473 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:26.066822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6b1e2-c8a7-492f-9cdd-3eab5b8ac473 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:26.068478 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6b1e2-c8a7-492f-9cdd-3eab5b8ac473 service swift] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:26.077351 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1174/2332] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jun 22 11:47:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 22 11:47:28.276084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8441762-8a6b-4352-ab3e-388113dd0fc4 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:28.277541 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f8441762-8a6b-4352-ab3e-388113dd0fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:28.292236 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:28.347169 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8441762-8a6b-4352-ab3e-388113dd0fc4 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:28.351745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b3b22-3065-4ea5-a35b-4dd6085fbafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:28.352672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b3b22-3065-4ea5-a35b-4dd6085fbafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:28.352901 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b3b22-3065-4ea5-a35b-4dd6085fbafc admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:28.353263 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b3b22-3065-4ea5-a35b-4dd6085fbafc admin admin] PATH_INFO: `/v3/users/6cd3a653cb514e37bad8080d8a5cc7b0` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:28.354491 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b3b22-3065-4ea5-a35b-4dd6085fbafc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cd3a653cb514e37bad8080d8a5cc7b0)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:28.356922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b3b22-3065-4ea5-a35b-4dd6085fbafc admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:28.408009 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-412b3b22-3065-4ea5-a35b-4dd6085fbafc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cd3a653cb514e37bad8080d8a5cc7b0'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:28.421508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-412b3b22-3065-4ea5-a35b-4dd6085fbafc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cd3a653cb514e37bad8080d8a5cc7b0'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:28.431921 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-412b3b22-3065-4ea5-a35b-4dd6085fbafc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cd3a653cb514e37bad8080d8a5cc7b0'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:28.432849 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-412b3b22-3065-4ea5-a35b-4dd6085fbafc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cd3a653cb514e37bad8080d8a5cc7b0'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:28.446634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-412b3b22-3065-4ea5-a35b-4dd6085fbafc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cd3a653cb514e37bad8080d8a5cc7b0'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:28.458856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1159/2333] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:47:28 2022] DELETE /identity/v3/users/6cd3a653cb514e37bad8080d8a5cc7b0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:47:28.460385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea034385-88f0-42a2-866a-e2974622eb70 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:28.460852 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea034385-88f0-42a2-866a-e2974622eb70 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:28.460996 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea034385-88f0-42a2-866a-e2974622eb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:28.490498 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-ea034385-88f0-42a2-866a-e2974622eb70 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:28.491771 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ea034385-88f0-42a2-866a-e2974622eb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:28.526390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:28.559029 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1175/2334] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:28.572848 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b3b22-3065-4ea5-a35b-4dd6085fbafc admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:28.574381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-412b3b22-3065-4ea5-a35b-4dd6085fbafc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:28.633655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b3b22-3065-4ea5-a35b-4dd6085fbafc admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:28.637971 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eff93d-8d97-422a-b285-2ab5094fa0b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:28.638626 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eff93d-8d97-422a-b285-2ab5094fa0b8 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:28.639473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eff93d-8d97-422a-b285-2ab5094fa0b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:28.639855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eff93d-8d97-422a-b285-2ab5094fa0b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:28.658046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eff93d-8d97-422a-b285-2ab5094fa0b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:28.658046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eff93d-8d97-422a-b285-2ab5094fa0b8 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:28.666869 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1160/2335] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:47:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:47:28.845516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea034385-88f0-42a2-866a-e2974622eb70 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:28.846997 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ea034385-88f0-42a2-866a-e2974622eb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:28.860152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea034385-88f0-42a2-866a-e2974622eb70 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:28.865607 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be6294-32eb-45e7-bb86-165d90675f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:28.866585 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be6294-32eb-45e7-bb86-165d90675f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:28.866893 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be6294-32eb-45e7-bb86-165d90675f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:28.867199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be6294-32eb-45e7-bb86-165d90675f9a admin admin] PATH_INFO: `/v3/projects/4829f904e10c4df197b253f1211157af` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:28.868810 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be6294-32eb-45e7-bb86-165d90675f9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4829f904e10c4df197b253f1211157af)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:28.870782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be6294-32eb-45e7-bb86-165d90675f9a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:28.920512 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-18be6294-32eb-45e7-bb86-165d90675f9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4829f904e10c4df197b253f1211157af', 'request_id': 'req-18be6294-32eb-45e7-bb86-165d90675f9a'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:28.927277 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-18be6294-32eb-45e7-bb86-165d90675f9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4829f904e10c4df197b253f1211157af', 'request_id': 'req-18be6294-32eb-45e7-bb86-165d90675f9a'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:28.935614 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-18be6294-32eb-45e7-bb86-165d90675f9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4829f904e10c4df197b253f1211157af', 'request_id': 'req-18be6294-32eb-45e7-bb86-165d90675f9a'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:28.958825 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-18be6294-32eb-45e7-bb86-165d90675f9a admin admin] The token cache is being invalidate because project 4829f904e10c4df197b253f1211157af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:47:28.959085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-18be6294-32eb-45e7-bb86-165d90675f9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:28.959815 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-18be6294-32eb-45e7-bb86-165d90675f9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:28.961640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1176/2336] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:47:28 2022] DELETE /identity/v3/projects/4829f904e10c4df197b253f1211157af => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:47:31.248759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eff93d-8d97-422a-b285-2ab5094fa0b8 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:31.250519 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-04eff93d-8d97-422a-b285-2ab5094fa0b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:31.269178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eff93d-8d97-422a-b285-2ab5094fa0b8 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:31.317544 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:31.366992 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277b5b3-cb9a-4194-a991-0b3228c86eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:31.368214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277b5b3-cb9a-4194-a991-0b3228c86eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:31.368465 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277b5b3-cb9a-4194-a991-0b3228c86eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:31.368816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277b5b3-cb9a-4194-a991-0b3228c86eb1 admin admin] PATH_INFO: `/v3/users/1123290f06914897b19017b49302e807` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:31.370434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277b5b3-cb9a-4194-a991-0b3228c86eb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1123290f06914897b19017b49302e807)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:31.372982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277b5b3-cb9a-4194-a991-0b3228c86eb1 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:31.452970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b277b5b3-cb9a-4194-a991-0b3228c86eb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1123290f06914897b19017b49302e807'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:31.468129 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b277b5b3-cb9a-4194-a991-0b3228c86eb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1123290f06914897b19017b49302e807'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:31.481983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b277b5b3-cb9a-4194-a991-0b3228c86eb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1123290f06914897b19017b49302e807'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:31.482880 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b277b5b3-cb9a-4194-a991-0b3228c86eb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1123290f06914897b19017b49302e807'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:31.492154 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b277b5b3-cb9a-4194-a991-0b3228c86eb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1123290f06914897b19017b49302e807'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:31.504754 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1161/2337] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:47:31 2022] DELETE /identity/v3/users/1123290f06914897b19017b49302e807 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:47:31.509316 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d43bea-058f-4d90-83e5-764a188d6566 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:31.509732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d43bea-058f-4d90-83e5-764a188d6566 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:31.509937 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d43bea-058f-4d90-83e5-764a188d6566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:31.531910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-20d43bea-058f-4d90-83e5-764a188d6566 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:31.533931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-20d43bea-058f-4d90-83e5-764a188d6566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:31.611647 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1177/2338] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:31.621949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277b5b3-cb9a-4194-a991-0b3228c86eb1 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:31.624039 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b277b5b3-cb9a-4194-a991-0b3228c86eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:31.706214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277b5b3-cb9a-4194-a991-0b3228c86eb1 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:31.710234 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ce7be-f99e-4516-80e1-5889d21ccbd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:31.711012 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ce7be-f99e-4516-80e1-5889d21ccbd1 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:31.711298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ce7be-f99e-4516-80e1-5889d21ccbd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:31.711770 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ce7be-f99e-4516-80e1-5889d21ccbd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:31.724501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ce7be-f99e-4516-80e1-5889d21ccbd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:31.726378 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ce7be-f99e-4516-80e1-5889d21ccbd1 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:31.739024 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1162/2339] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:47:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:47:32.213860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d43bea-058f-4d90-83e5-764a188d6566 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:32.215453 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-20d43bea-058f-4d90-83e5-764a188d6566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:32.231483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d43bea-058f-4d90-83e5-764a188d6566 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:32.237305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd814f3-8f99-43c9-9b16-0735277bd62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:32.238305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd814f3-8f99-43c9-9b16-0735277bd62d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:32.238670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd814f3-8f99-43c9-9b16-0735277bd62d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:32.239042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd814f3-8f99-43c9-9b16-0735277bd62d admin admin] PATH_INFO: `/v3/projects/d0de877c39ed464fb198fc135518407c` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:32.240434 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd814f3-8f99-43c9-9b16-0735277bd62d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0de877c39ed464fb198fc135518407c)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:32.242554 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd814f3-8f99-43c9-9b16-0735277bd62d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:32.303701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8fd814f3-8f99-43c9-9b16-0735277bd62d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0de877c39ed464fb198fc135518407c', 'request_id': 'req-8fd814f3-8f99-43c9-9b16-0735277bd62d'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:32.316001 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8fd814f3-8f99-43c9-9b16-0735277bd62d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0de877c39ed464fb198fc135518407c', 'request_id': 'req-8fd814f3-8f99-43c9-9b16-0735277bd62d'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:32.318109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:32.325193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8fd814f3-8f99-43c9-9b16-0735277bd62d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0de877c39ed464fb198fc135518407c', 'request_id': 'req-8fd814f3-8f99-43c9-9b16-0735277bd62d'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:32.337757 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8fd814f3-8f99-43c9-9b16-0735277bd62d admin admin] The token cache is being invalidate because project d0de877c39ed464fb198fc135518407c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:47:32.338077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8fd814f3-8f99-43c9-9b16-0735277bd62d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:32.338724 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8fd814f3-8f99-43c9-9b16-0735277bd62d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:32.340648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1178/2340] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:47:32 2022] DELETE /identity/v3/projects/d0de877c39ed464fb198fc135518407c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:47:32.346843 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbed81-4ab1-40ff-a56b-1a31ac32b537 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:32.347589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbed81-4ab1-40ff-a56b-1a31ac32b537 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:32.347589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dbed81-4ab1-40ff-a56b-1a31ac32b537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:32.366859 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-d6dbed81-4ab1-40ff-a56b-1a31ac32b537 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:32.368127 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d6dbed81-4ab1-40ff-a56b-1a31ac32b537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:32.420463 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:32.435071 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1163/2341] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:32.442927 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b92789-1be2-40a4-a24b-abad00f3039c None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:32.443322 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b92789-1be2-40a4-a24b-abad00f3039c None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:32.443780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b92789-1be2-40a4-a24b-abad00f3039c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:32.463565 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-78b92789-1be2-40a4-a24b-abad00f3039c None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:32.465187 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-78b92789-1be2-40a4-a24b-abad00f3039c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:32.480488 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1179/2342] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:32.487738 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dbed81-4ab1-40ff-a56b-1a31ac32b537 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:32.511143 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dbed81-4ab1-40ff-a56b-1a31ac32b537 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:32.521962 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fec952-9f2c-48fa-ba2a-1e6e41e6180c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:32.523997 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fec952-9f2c-48fa-ba2a-1e6e41e6180c admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:32.524724 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fec952-9f2c-48fa-ba2a-1e6e41e6180c admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:32.525488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fec952-9f2c-48fa-ba2a-1e6e41e6180c admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:32.527690 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fec952-9f2c-48fa-ba2a-1e6e41e6180c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:47:32.528478 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fec952-9f2c-48fa-ba2a-1e6e41e6180c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:32.533749 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fec952-9f2c-48fa-ba2a-1e6e41e6180c admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:32.561663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1164/2343] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:47:32.562590 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b92789-1be2-40a4-a24b-abad00f3039c None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:32.572368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b92789-1be2-40a4-a24b-abad00f3039c None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:32.578095 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56f8e6-dc99-4881-bbbf-aa620c8227b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:32.578989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56f8e6-dc99-4881-bbbf-aa620c8227b6 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:32.579494 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56f8e6-dc99-4881-bbbf-aa620c8227b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:32.580139 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56f8e6-dc99-4881-bbbf-aa620c8227b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:32.581271 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56f8e6-dc99-4881-bbbf-aa620c8227b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:32.584701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c56f8e6-dc99-4881-bbbf-aa620c8227b6 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:32.620071 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1180/2344] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:47:32 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:47:32.625186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fec952-9f2c-48fa-ba2a-1e6e41e6180c admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:32.637333 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fec952-9f2c-48fa-ba2a-1e6e41e6180c admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:32.713283 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3b9e1-b7be-4cce-810f-f391c1a9d8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:32.714007 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3b9e1-b7be-4cce-810f-f391c1a9d8ae admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:32.714241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3b9e1-b7be-4cce-810f-f391c1a9d8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:32.714518 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3b9e1-b7be-4cce-810f-f391c1a9d8ae admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:32.715273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3b9e1-b7be-4cce-810f-f391c1a9d8ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:32.717263 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3b9e1-b7be-4cce-810f-f391c1a9d8ae admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:32.741576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1165/2345] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:47:32 2022] POST /identity/v3/users => generated 286 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:47:32.746037 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56f8e6-dc99-4881-bbbf-aa620c8227b6 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:32.759623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56f8e6-dc99-4881-bbbf-aa620c8227b6 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:32.765851 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c8581-9682-4ea4-bcab-395c97c39a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:32.767546 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c8581-9682-4ea4-bcab-395c97c39a54 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:32.768040 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c8581-9682-4ea4-bcab-395c97c39a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:32.768625 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c8581-9682-4ea4-bcab-395c97c39a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:32.769869 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c8581-9682-4ea4-bcab-395c97c39a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:32.771622 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c8581-9682-4ea4-bcab-395c97c39a54 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:32.799848 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1181/2346] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:47:32.804810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3b9e1-b7be-4cce-810f-f391c1a9d8ae admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:32.816084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3b9e1-b7be-4cce-810f-f391c1a9d8ae admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:32.821167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdad1af-cb81-43e0-8e08-c7bcf7dad033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:32.821944 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdad1af-cb81-43e0-8e08-c7bcf7dad033 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:32.822252 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdad1af-cb81-43e0-8e08-c7bcf7dad033 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:32.822528 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdad1af-cb81-43e0-8e08-c7bcf7dad033 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:32.823249 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdad1af-cb81-43e0-8e08-c7bcf7dad033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:32.825102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdad1af-cb81-43e0-8e08-c7bcf7dad033 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:32.842997 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1166/2347] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:47:32.844178 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c8581-9682-4ea4-bcab-395c97c39a54 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:32.854194 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c8581-9682-4ea4-bcab-395c97c39a54 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:32.858919 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f01d7-e00c-4ed8-a653-8fc3d290e4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:32.859833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f01d7-e00c-4ed8-a653-8fc3d290e4ce admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:32.860116 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f01d7-e00c-4ed8-a653-8fc3d290e4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:32.860393 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f01d7-e00c-4ed8-a653-8fc3d290e4ce admin admin] PATH_INFO: `/v3/projects/4845f1df49c64dcc8c4a48691fae9a27/users/58976549233e4bb6991258f7da1123ae/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:32.876976 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f01d7-e00c-4ed8-a653-8fc3d290e4ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=4845f1df49c64dcc8c4a48691fae9a27, user_id=58976549233e4bb6991258f7da1123ae, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:32.879693 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f01d7-e00c-4ed8-a653-8fc3d290e4ce admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:32.894307 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1182/2348] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:47:32 2022] PUT /identity/v3/projects/4845f1df49c64dcc8c4a48691fae9a27/users/58976549233e4bb6991258f7da1123ae/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:47:32.899262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ad9e3-a693-4ac1-ae28-27e278d352c3 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:32.899577 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ad9e3-a693-4ac1-ae28-27e278d352c3 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:32.899858 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ad9e3-a693-4ac1-ae28-27e278d352c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:32.917653 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-492ad9e3-a693-4ac1-ae28-27e278d352c3 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:32.918891 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-492ad9e3-a693-4ac1-ae28-27e278d352c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:33.000390 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1167/2349] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:33.011916 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f01d7-e00c-4ed8-a653-8fc3d290e4ce admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:33.013776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d35f01d7-e00c-4ed8-a653-8fc3d290e4ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:33.086373 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f01d7-e00c-4ed8-a653-8fc3d290e4ce admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:33.090328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10993ef-8833-4ae6-9a8e-e04cfe0bc378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:33.091073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10993ef-8833-4ae6-9a8e-e04cfe0bc378 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:33.091452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10993ef-8833-4ae6-9a8e-e04cfe0bc378 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:33.091862 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10993ef-8833-4ae6-9a8e-e04cfe0bc378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:33.105616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10993ef-8833-4ae6-9a8e-e04cfe0bc378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:33.108064 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10993ef-8833-4ae6-9a8e-e04cfe0bc378 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:33.119415 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1183/2350] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:47:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:47:37.616571 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e1e87-3708-4aba-81f5-c1787bb56eb3 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:37.617136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e1e87-3708-4aba-81f5-c1787bb56eb3 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:37.617136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e1e87-3708-4aba-81f5-c1787bb56eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:37.660910 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c55e1e87-3708-4aba-81f5-c1787bb56eb3 None None] MFA Rules not processed for user `58976549233e4bb6991258f7da1123ae`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:37.662156 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c55e1e87-3708-4aba-81f5-c1787bb56eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:37.742763 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:37.804615 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1168/2351] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:37.805742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10993ef-8833-4ae6-9a8e-e04cfe0bc378 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:37.869122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10993ef-8833-4ae6-9a8e-e04cfe0bc378 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:37.874731 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600bf72-e1f8-4107-921a-2e5c2320a8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:37.875421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600bf72-e1f8-4107-921a-2e5c2320a8b0 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:37.875710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600bf72-e1f8-4107-921a-2e5c2320a8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:37.875987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600bf72-e1f8-4107-921a-2e5c2320a8b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:37.876725 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600bf72-e1f8-4107-921a-2e5c2320a8b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:37.878651 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600bf72-e1f8-4107-921a-2e5c2320a8b0 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:37.921268 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1184/2352] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:47:37 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:47:37.925590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e1e87-3708-4aba-81f5-c1787bb56eb3 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:37.938389 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e1e87-3708-4aba-81f5-c1787bb56eb3 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:38.001003 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717180a9-bc26-417e-9c35-db5cad48d3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:38.001680 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717180a9-bc26-417e-9c35-db5cad48d3d9 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:38.001966 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717180a9-bc26-417e-9c35-db5cad48d3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:38.002243 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717180a9-bc26-417e-9c35-db5cad48d3d9 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:38.002934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717180a9-bc26-417e-9c35-db5cad48d3d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:38.004915 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717180a9-bc26-417e-9c35-db5cad48d3d9 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:38.031454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1169/2353] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:47:37 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:47:38.032896 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600bf72-e1f8-4107-921a-2e5c2320a8b0 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:38.043737 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600bf72-e1f8-4107-921a-2e5c2320a8b0 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:38.048396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ed686-2b4f-4b88-b6da-b6a91795c9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:38.049063 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ed686-2b4f-4b88-b6da-b6a91795c9ac admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:38.049346 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ed686-2b4f-4b88-b6da-b6a91795c9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:38.049620 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ed686-2b4f-4b88-b6da-b6a91795c9ac admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:38.050342 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ed686-2b4f-4b88-b6da-b6a91795c9ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:38.052018 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ed686-2b4f-4b88-b6da-b6a91795c9ac admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:38.054660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:38.068723 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1185/2354] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:47:38.072654 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717180a9-bc26-417e-9c35-db5cad48d3d9 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:38.085439 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717180a9-bc26-417e-9c35-db5cad48d3d9 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:38.094104 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d314090-a98b-45f2-9868-fbe99e378a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:38.095264 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d314090-a98b-45f2-9868-fbe99e378a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:38.095686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d314090-a98b-45f2-9868-fbe99e378a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:38.096063 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d314090-a98b-45f2-9868-fbe99e378a68 admin admin] PATH_INFO: `/v3/projects/97aa107a7cc34f0e8d55b4586f339fbe/users/1ab42aa23f0947479254a5f79aaa226c/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:38.114863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d314090-a98b-45f2-9868-fbe99e378a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97aa107a7cc34f0e8d55b4586f339fbe, user_id=1ab42aa23f0947479254a5f79aaa226c, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:38.117833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d314090-a98b-45f2-9868-fbe99e378a68 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:38.134231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1170/2355] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:47:38 2022] PUT /identity/v3/projects/97aa107a7cc34f0e8d55b4586f339fbe/users/1ab42aa23f0947479254a5f79aaa226c/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:47:38.138222 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ed686-2b4f-4b88-b6da-b6a91795c9ac admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:38.149863 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ed686-2b4f-4b88-b6da-b6a91795c9ac admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:38.219548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f40779-75af-444a-a1f5-54e8999cc0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:38.222410 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f40779-75af-444a-a1f5-54e8999cc0f7 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:38.222695 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f40779-75af-444a-a1f5-54e8999cc0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:38.222970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f40779-75af-444a-a1f5-54e8999cc0f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:38.223732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f40779-75af-444a-a1f5-54e8999cc0f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:38.225377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f40779-75af-444a-a1f5-54e8999cc0f7 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:38.236374 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1186/2356] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:47:38.240507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d314090-a98b-45f2-9868-fbe99e378a68 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:38.264413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d314090-a98b-45f2-9868-fbe99e378a68 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:38.274984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2edae54-dcbd-48ca-83da-fb003101e2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:38.275945 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edae54-dcbd-48ca-83da-fb003101e2b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:38.276226 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edae54-dcbd-48ca-83da-fb003101e2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:38.276502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2edae54-dcbd-48ca-83da-fb003101e2b3 admin admin] PATH_INFO: `/v3/projects/97aa107a7cc34f0e8d55b4586f339fbe/users/1ab42aa23f0947479254a5f79aaa226c/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:38.278638 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2edae54-dcbd-48ca-83da-fb003101e2b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97aa107a7cc34f0e8d55b4586f339fbe, user_id=1ab42aa23f0947479254a5f79aaa226c, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:38.282041 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2edae54-dcbd-48ca-83da-fb003101e2b3 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:38.293253 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1171/2357] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:47:38 2022] PUT /identity/v3/projects/97aa107a7cc34f0e8d55b4586f339fbe/users/1ab42aa23f0947479254a5f79aaa226c/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:47:39.940587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f40779-75af-444a-a1f5-54e8999cc0f7 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:39.948129 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-24f40779-75af-444a-a1f5-54e8999cc0f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:39.965865 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f40779-75af-444a-a1f5-54e8999cc0f7 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:40.003220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3651737-f482-4fe1-a706-b26020a5f515 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:40.004143 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3651737-f482-4fe1-a706-b26020a5f515 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:40.004817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3651737-f482-4fe1-a706-b26020a5f515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:40.037586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c3651737-f482-4fe1-a706-b26020a5f515 None None] MFA Rules not processed for user `955c44ffe1bd426e900c7405df44d1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:40.038743 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c3651737-f482-4fe1-a706-b26020a5f515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:40.079289 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f11dfd-76fe-4b4c-a5ac-5024732f2092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:40.080500 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f11dfd-76fe-4b4c-a5ac-5024732f2092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:40.080925 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f11dfd-76fe-4b4c-a5ac-5024732f2092 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:40.081340 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f11dfd-76fe-4b4c-a5ac-5024732f2092 admin admin] PATH_INFO: `/v3/users/34c9ae5b04924677a19d1e3bc6247d47` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:40.083061 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f11dfd-76fe-4b4c-a5ac-5024732f2092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34c9ae5b04924677a19d1e3bc6247d47)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:40.085286 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f11dfd-76fe-4b4c-a5ac-5024732f2092 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:40.191045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-81f11dfd-76fe-4b4c-a5ac-5024732f2092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34c9ae5b04924677a19d1e3bc6247d47'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:40.202655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1172/2358] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jun 22 11:47:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:47:40.221888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3651737-f482-4fe1-a706-b26020a5f515 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:40.224190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c3651737-f482-4fe1-a706-b26020a5f515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:40.225889 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-81f11dfd-76fe-4b4c-a5ac-5024732f2092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34c9ae5b04924677a19d1e3bc6247d47'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:40.240303 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-81f11dfd-76fe-4b4c-a5ac-5024732f2092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34c9ae5b04924677a19d1e3bc6247d47'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:40.250013 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-81f11dfd-76fe-4b4c-a5ac-5024732f2092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34c9ae5b04924677a19d1e3bc6247d47'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:40.251106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-81f11dfd-76fe-4b4c-a5ac-5024732f2092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34c9ae5b04924677a19d1e3bc6247d47'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:40.257485 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:40.268835 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1187/2359] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:47:39 2022] DELETE /identity/v3/users/34c9ae5b04924677a19d1e3bc6247d47 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:47:40.272868 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5012be-7eb9-41a1-8915-11347fd3becc None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:40.273163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5012be-7eb9-41a1-8915-11347fd3becc None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:40.273602 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5012be-7eb9-41a1-8915-11347fd3becc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:40.315390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:40.319994 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-4b5012be-7eb9-41a1-8915-11347fd3becc None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:40.322069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-4b5012be-7eb9-41a1-8915-11347fd3becc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:40.358197 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3651737-f482-4fe1-a706-b26020a5f515 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:40.431967 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1188/2360] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:40.442881 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5012be-7eb9-41a1-8915-11347fd3becc None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:40.446437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-4b5012be-7eb9-41a1-8915-11347fd3becc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:40.468378 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b2745-5680-4e5c-b728-65b3b6b43bf9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd642e290e44e67b56eb3b8361d7af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:40.469523 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b2745-5680-4e5c-b728-65b3b6b43bf9 service swift] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:40.469826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b2745-5680-4e5c-b728-65b3b6b43bf9 service swift] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:40.470201 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b2745-5680-4e5c-b728-65b3b6b43bf9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:40.474197 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-da9b2745-5680-4e5c-b728-65b3b6b43bf9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:40.570951 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b2745-5680-4e5c-b728-65b3b6b43bf9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:40.573137 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b2745-5680-4e5c-b728-65b3b6b43bf9 service swift] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:40.576830 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5012be-7eb9-41a1-8915-11347fd3becc None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:40.584762 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1173/2361] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jun 22 11:47:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 22 11:47:40.586997 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74347d3-377b-4b6d-b593-25a98195b93f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:40.587757 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74347d3-377b-4b6d-b593-25a98195b93f service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:40.588082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74347d3-377b-4b6d-b593-25a98195b93f service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:40.588377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74347d3-377b-4b6d-b593-25a98195b93f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:40.600077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74347d3-377b-4b6d-b593-25a98195b93f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:40.601913 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74347d3-377b-4b6d-b593-25a98195b93f service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:40.617701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1189/2362] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:47:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:47:41.460684 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b2745-5680-4e5c-b728-65b3b6b43bf9 service swift] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:41.462374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-da9b2745-5680-4e5c-b728-65b3b6b43bf9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:41.510449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b2745-5680-4e5c-b728-65b3b6b43bf9 service swift] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:41.536510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed61c17-19c6-427b-9e9f-2e1135872f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:41.539807 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed61c17-19c6-427b-9e9f-2e1135872f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:41.540374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed61c17-19c6-427b-9e9f-2e1135872f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:41.541430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed61c17-19c6-427b-9e9f-2e1135872f86 admin admin] PATH_INFO: `/v3/projects/3c78f985b56446978713e0645381509e` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:41.545379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed61c17-19c6-427b-9e9f-2e1135872f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c78f985b56446978713e0645381509e)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:41.553830 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed61c17-19c6-427b-9e9f-2e1135872f86 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:41.762772 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:41.766092 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8ed61c17-19c6-427b-9e9f-2e1135872f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c78f985b56446978713e0645381509e', 'request_id': 'req-8ed61c17-19c6-427b-9e9f-2e1135872f86'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:41.789314 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8ed61c17-19c6-427b-9e9f-2e1135872f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c78f985b56446978713e0645381509e', 'request_id': 'req-8ed61c17-19c6-427b-9e9f-2e1135872f86'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:41.817036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8ed61c17-19c6-427b-9e9f-2e1135872f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c78f985b56446978713e0645381509e', 'request_id': 'req-8ed61c17-19c6-427b-9e9f-2e1135872f86'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:41.860310 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8ed61c17-19c6-427b-9e9f-2e1135872f86 admin admin] The token cache is being invalidate because project 3c78f985b56446978713e0645381509e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:47:41.860637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8ed61c17-19c6-427b-9e9f-2e1135872f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:41.864231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8ed61c17-19c6-427b-9e9f-2e1135872f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:47:41.872543 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1174/2363] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:47:41 2022] DELETE /identity/v3/projects/3c78f985b56446978713e0645381509e => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:47:41.880213 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4c390-1251-4ca7-9bf9-6446a7de00a3 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:41.882657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4c390-1251-4ca7-9bf9-6446a7de00a3 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:41.883757 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b4c390-1251-4ca7-9bf9-6446a7de00a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:42.082533 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-78b4c390-1251-4ca7-9bf9-6446a7de00a3 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:42.085885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-78b4c390-1251-4ca7-9bf9-6446a7de00a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:42.269011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1190/2364] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:42.271691 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bd27a-8528-49fc-9612-e5129a0adf61 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:42.272335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bd27a-8528-49fc-9612-e5129a0adf61 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:42.272774 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bd27a-8528-49fc-9612-e5129a0adf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:42.320283 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-343bd27a-8528-49fc-9612-e5129a0adf61 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:42.322150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-343bd27a-8528-49fc-9612-e5129a0adf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:42.344999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1175/2365] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:42.349902 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4c390-1251-4ca7-9bf9-6446a7de00a3 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:42.365764 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b4c390-1251-4ca7-9bf9-6446a7de00a3 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:42.383117 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091ecac-4fa1-4847-b2d1-b49d8ffdaee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:42.384209 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091ecac-4fa1-4847-b2d1-b49d8ffdaee2 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:42.384682 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091ecac-4fa1-4847-b2d1-b49d8ffdaee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:42.385712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091ecac-4fa1-4847-b2d1-b49d8ffdaee2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:42.386632 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091ecac-4fa1-4847-b2d1-b49d8ffdaee2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:47:42.387055 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091ecac-4fa1-4847-b2d1-b49d8ffdaee2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:42.393365 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091ecac-4fa1-4847-b2d1-b49d8ffdaee2 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:42.430500 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1191/2366] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:47:42.431830 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bd27a-8528-49fc-9612-e5129a0adf61 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:42.445501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bd27a-8528-49fc-9612-e5129a0adf61 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:42.459117 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c51b03-5c4d-4dc3-a1da-be3bc19c89e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:42.460093 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c51b03-5c4d-4dc3-a1da-be3bc19c89e8 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:42.460615 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c51b03-5c4d-4dc3-a1da-be3bc19c89e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:42.461051 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c51b03-5c4d-4dc3-a1da-be3bc19c89e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:42.463530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c51b03-5c4d-4dc3-a1da-be3bc19c89e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:42.475067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c51b03-5c4d-4dc3-a1da-be3bc19c89e8 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:42.571551 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1176/2367] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:47:42 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:47:42.572530 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091ecac-4fa1-4847-b2d1-b49d8ffdaee2 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:42.578619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:42.592177 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091ecac-4fa1-4847-b2d1-b49d8ffdaee2 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:42.733998 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ceda9e-2068-4e5c-b725-c70feee40645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:42.735689 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ceda9e-2068-4e5c-b725-c70feee40645 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:42.736095 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ceda9e-2068-4e5c-b725-c70feee40645 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:42.736496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ceda9e-2068-4e5c-b725-c70feee40645 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:42.738767 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ceda9e-2068-4e5c-b725-c70feee40645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:42.740949 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ceda9e-2068-4e5c-b725-c70feee40645 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:42.790985 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1192/2368] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:47:42 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:47:42.793231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c51b03-5c4d-4dc3-a1da-be3bc19c89e8 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:42.816047 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c51b03-5c4d-4dc3-a1da-be3bc19c89e8 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:42.828041 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fb920-d870-460c-a4e7-346b9313fe46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:42.828736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fb920-d870-460c-a4e7-346b9313fe46 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:42.829018 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fb920-d870-460c-a4e7-346b9313fe46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:42.829296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fb920-d870-460c-a4e7-346b9313fe46 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:42.833739 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fb920-d870-460c-a4e7-346b9313fe46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:42.836511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fb920-d870-460c-a4e7-346b9313fe46 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:42.866742 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1177/2369] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:47:42 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:47:42.868173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ceda9e-2068-4e5c-b725-c70feee40645 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:42.887074 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ceda9e-2068-4e5c-b725-c70feee40645 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:42.907885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94aa825-e9f2-4bbf-a0bb-3863bce8d7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:42.908764 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94aa825-e9f2-4bbf-a0bb-3863bce8d7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:42.909190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94aa825-e9f2-4bbf-a0bb-3863bce8d7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:42.909627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94aa825-e9f2-4bbf-a0bb-3863bce8d7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:42.910508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94aa825-e9f2-4bbf-a0bb-3863bce8d7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:42.915207 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94aa825-e9f2-4bbf-a0bb-3863bce8d7a9 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:42.951733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1193/2370] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:47:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:47:42.954150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fb920-d870-460c-a4e7-346b9313fe46 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:42.967408 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fb920-d870-460c-a4e7-346b9313fe46 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:42.992015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa91a99-eaa6-4032-ad37-bc3df8a1db16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:42.997124 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa91a99-eaa6-4032-ad37-bc3df8a1db16 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:42.999104 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa91a99-eaa6-4032-ad37-bc3df8a1db16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:42.999604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa91a99-eaa6-4032-ad37-bc3df8a1db16 admin admin] PATH_INFO: `/v3/projects/2975e2c648284a80824126eecc5d0269/users/8fc6cd29af4d4c28bec9bf6cc9db5749/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:43.042313 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa91a99-eaa6-4032-ad37-bc3df8a1db16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2975e2c648284a80824126eecc5d0269, user_id=8fc6cd29af4d4c28bec9bf6cc9db5749, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:43.053671 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa91a99-eaa6-4032-ad37-bc3df8a1db16 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:43.097936 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1178/2371] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:47:42 2022] PUT /identity/v3/projects/2975e2c648284a80824126eecc5d0269/users/8fc6cd29af4d4c28bec9bf6cc9db5749/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:47:43.103376 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6daa7-f199-4030-9120-605867ff28fd None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:43.105215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6daa7-f199-4030-9120-605867ff28fd None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:43.105822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6daa7-f199-4030-9120-605867ff28fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:43.140672 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f2e6daa7-f199-4030-9120-605867ff28fd None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:43.143664 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f2e6daa7-f199-4030-9120-605867ff28fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:43.314294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1194/2372] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:47:43.315284 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa91a99-eaa6-4032-ad37-bc3df8a1db16 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:43.317067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-6aa91a99-eaa6-4032-ad37-bc3df8a1db16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:43.480436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa91a99-eaa6-4032-ad37-bc3df8a1db16 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:43.499949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff97882-5c3c-4d35-abb7-fa04640ea751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:43.500726 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff97882-5c3c-4d35-abb7-fa04640ea751 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:43.503617 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff97882-5c3c-4d35-abb7-fa04640ea751 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:43.503928 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff97882-5c3c-4d35-abb7-fa04640ea751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:43.547706 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff97882-5c3c-4d35-abb7-fa04640ea751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:43.554048 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff97882-5c3c-4d35-abb7-fa04640ea751 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:43.575153 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1179/2373] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:47:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:47:44.965591 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437621cd-5473-474d-bbe1-540bdd965d3e None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:44.966955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437621cd-5473-474d-bbe1-540bdd965d3e None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:44.967424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437621cd-5473-474d-bbe1-540bdd965d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:45.012031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-437621cd-5473-474d-bbe1-540bdd965d3e None None] MFA Rules not processed for user `1ab42aa23f0947479254a5f79aaa226c`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:45.013463 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-437621cd-5473-474d-bbe1-540bdd965d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:45.119735 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1195/2374] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:45.144918 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff97882-5c3c-4d35-abb7-fa04640ea751 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:45.149217 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-dff97882-5c3c-4d35-abb7-fa04640ea751 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:45.342448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff97882-5c3c-4d35-abb7-fa04640ea751 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:45.372029 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183eaa1d-973b-4d3b-800e-fb4d1389bcf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:45.376564 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183eaa1d-973b-4d3b-800e-fb4d1389bcf9 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:45.378062 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183eaa1d-973b-4d3b-800e-fb4d1389bcf9 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:45.378665 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183eaa1d-973b-4d3b-800e-fb4d1389bcf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:45.381104 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-183eaa1d-973b-4d3b-800e-fb4d1389bcf9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:45.399198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:45.525979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183eaa1d-973b-4d3b-800e-fb4d1389bcf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:45.528043 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183eaa1d-973b-4d3b-800e-fb4d1389bcf9 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:45.566996 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1180/2375] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:47:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:47:49.489694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3e26d-ad3b-4338-8a73-72b1f0cf7777 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:49.491701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3e26d-ad3b-4338-8a73-72b1f0cf7777 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:49.492147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3e26d-ad3b-4338-8a73-72b1f0cf7777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:49.514173 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:49.532257 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-ddd3e26d-ad3b-4338-8a73-72b1f0cf7777 None None] MFA Rules not processed for user `27cc3d38926f470c91eeb6aecfc86a48`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:49.533578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ddd3e26d-ad3b-4338-8a73-72b1f0cf7777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:49.699476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1196/2376] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:49.870850 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c5aa1-bba6-4386-ac07-817329fe6d73 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:49.876075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c5aa1-bba6-4386-ac07-817329fe6d73 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:49.876636 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c5aa1-bba6-4386-ac07-817329fe6d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:49.915056 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-5e1c5aa1-bba6-4386-ac07-817329fe6d73 None None] MFA Rules not processed for user `58976549233e4bb6991258f7da1123ae`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:49.925698 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-5e1c5aa1-bba6-4386-ac07-817329fe6d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:49.958024 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1181/2377] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:47:52.385884 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9e14a-1a81-497b-9376-f49f6af0b315 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:52.387547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9e14a-1a81-497b-9376-f49f6af0b315 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:52.388082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9e14a-1a81-497b-9376-f49f6af0b315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:52.436075 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-37c9e14a-1a81-497b-9376-f49f6af0b315 None None] MFA Rules not processed for user `8fc6cd29af4d4c28bec9bf6cc9db5749`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:47:52.442067 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-37c9e14a-1a81-497b-9376-f49f6af0b315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:47:52.532661 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1197/2378] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:47:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:47:52.537697 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c5aa1-bba6-4386-ac07-817329fe6d73 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:52.552036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c5aa1-bba6-4386-ac07-817329fe6d73 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:52.557304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c261f-6947-48b3-8453-63701d4e9afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:52.558185 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c261f-6947-48b3-8453-63701d4e9afa admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:52.558625 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c261f-6947-48b3-8453-63701d4e9afa admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:52.559696 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c261f-6947-48b3-8453-63701d4e9afa admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:52.560640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c261f-6947-48b3-8453-63701d4e9afa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:52.562723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c261f-6947-48b3-8453-63701d4e9afa admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:52.663148 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9e14a-1a81-497b-9376-f49f6af0b315 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:52.663856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1182/2379] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:47:52 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:47:52.682541 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9e14a-1a81-497b-9376-f49f6af0b315 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:52.839267 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0eba44-e8c4-4743-843e-3cf6bde3938c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:52.840808 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0eba44-e8c4-4743-843e-3cf6bde3938c admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:52.841823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0eba44-e8c4-4743-843e-3cf6bde3938c admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:52.842263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0eba44-e8c4-4743-843e-3cf6bde3938c admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:52.843147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0eba44-e8c4-4743-843e-3cf6bde3938c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:52.845984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0eba44-e8c4-4743-843e-3cf6bde3938c admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:52.900210 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1198/2380] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:47:52 2022] POST /identity/v3/users => generated 299 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:47:52.904916 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c261f-6947-48b3-8453-63701d4e9afa admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:52.929215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c261f-6947-48b3-8453-63701d4e9afa admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:52.937403 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e581f-f9e3-4c56-b7e6-50fa55c1f761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:52.938472 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e581f-f9e3-4c56-b7e6-50fa55c1f761 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:52.938914 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e581f-f9e3-4c56-b7e6-50fa55c1f761 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:52.939396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e581f-f9e3-4c56-b7e6-50fa55c1f761 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:52.940294 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e581f-f9e3-4c56-b7e6-50fa55c1f761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:52.942337 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e581f-f9e3-4c56-b7e6-50fa55c1f761 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:52.960034 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1183/2381] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:47:52.964415 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0eba44-e8c4-4743-843e-3cf6bde3938c admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:52.979977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0eba44-e8c4-4743-843e-3cf6bde3938c admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:52.990395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d238a9f-7013-4e8f-b338-c4837c675a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:52.993029 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d238a9f-7013-4e8f-b338-c4837c675a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:52.998465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d238a9f-7013-4e8f-b338-c4837c675a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:52.998735 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d238a9f-7013-4e8f-b338-c4837c675a56 admin admin] PATH_INFO: `/v3/projects/e7f491d21b9f4d2795facfff24b32526/users/8819384277834fe795cc0017d1ff29c4/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:53.037095 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d238a9f-7013-4e8f-b338-c4837c675a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7f491d21b9f4d2795facfff24b32526, user_id=8819384277834fe795cc0017d1ff29c4, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:53.041441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d238a9f-7013-4e8f-b338-c4837c675a56 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:53.060803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1199/2382] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:47:52 2022] PUT /identity/v3/projects/e7f491d21b9f4d2795facfff24b32526/users/8819384277834fe795cc0017d1ff29c4/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:47:53.065494 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e581f-f9e3-4c56-b7e6-50fa55c1f761 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:53.084058 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e581f-f9e3-4c56-b7e6-50fa55c1f761 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:53.206584 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab6831-ff23-4823-9d14-ec242d7e3456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:53.207381 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab6831-ff23-4823-9d14-ec242d7e3456 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:53.208398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab6831-ff23-4823-9d14-ec242d7e3456 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:53.208398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab6831-ff23-4823-9d14-ec242d7e3456 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:53.208852 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab6831-ff23-4823-9d14-ec242d7e3456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:53.211152 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab6831-ff23-4823-9d14-ec242d7e3456 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:53.229843 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1184/2383] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:47:53 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:47:53.236382 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d238a9f-7013-4e8f-b338-c4837c675a56 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:47:53.253302 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d238a9f-7013-4e8f-b338-c4837c675a56 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:47:53.265776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5b81a-d389-464c-b123-e1f5c3304b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:47:53.266887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5b81a-d389-464c-b123-e1f5c3304b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:47:53.267180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5b81a-d389-464c-b123-e1f5c3304b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:47:53.268054 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5b81a-d389-464c-b123-e1f5c3304b8a admin admin] PATH_INFO: `/v3/projects/e7f491d21b9f4d2795facfff24b32526/users/8819384277834fe795cc0017d1ff29c4/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:47:53.270656 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5b81a-d389-464c-b123-e1f5c3304b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7f491d21b9f4d2795facfff24b32526, user_id=8819384277834fe795cc0017d1ff29c4, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:47:53.273484 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5b81a-d389-464c-b123-e1f5c3304b8a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:47:53.276582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:47:53.321519 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1200/2384] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:47:53 2022] PUT /identity/v3/projects/e7f491d21b9f4d2795facfff24b32526/users/8819384277834fe795cc0017d1ff29c4/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:48:02.777351 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8e995-f443-4513-82b8-87a1114b17b0 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:02.777920 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8e995-f443-4513-82b8-87a1114b17b0 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:02.777920 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8e995-f443-4513-82b8-87a1114b17b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:02.820402 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-66b8e995-f443-4513-82b8-87a1114b17b0 None None] MFA Rules not processed for user `8819384277834fe795cc0017d1ff29c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:48:02.822714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-66b8e995-f443-4513-82b8-87a1114b17b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:48:03.002110 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:48:03.021036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1185/2385] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:48:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:48:03.309099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:INSERT=1,keystone:SELECT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:48:06.405616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5b81a-d389-464c-b123-e1f5c3304b8a admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:48:06.408157 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-2cc5b81a-d389-464c-b123-e1f5c3304b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:48:06.490436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5b81a-d389-464c-b123-e1f5c3304b8a admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:48:06.494985 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae465f51-d587-4373-947d-da1b273b44a8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ed133721ed044708ee362c93f564d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:48:06.496017 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae465f51-d587-4373-947d-da1b273b44a8 service glance] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:06.496530 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae465f51-d587-4373-947d-da1b273b44a8 service glance] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:06.513993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae465f51-d587-4373-947d-da1b273b44a8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:06.598483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae465f51-d587-4373-947d-da1b273b44a8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:48:06.600120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae465f51-d587-4373-947d-da1b273b44a8 service glance] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:48:06.612072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1201/2386] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:48:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:48:06.890443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b20e8-404b-45f0-a70c-ca186dce6ff3 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:06.891055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b20e8-404b-45f0-a70c-ca186dce6ff3 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:06.891055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b20e8-404b-45f0-a70c-ca186dce6ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:06.962976 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-8d5b20e8-404b-45f0-a70c-ca186dce6ff3 None None] MFA Rules not processed for user `955c44ffe1bd426e900c7405df44d1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:48:06.964307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-8d5b20e8-404b-45f0-a70c-ca186dce6ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:48:07.144671 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1186/2387] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jun 22 11:48:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:48:07.160334 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae465f51-d587-4373-947d-da1b273b44a8 service glance] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:48:07.161834 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ae465f51-d587-4373-947d-da1b273b44a8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:48:07.381760 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae465f51-d587-4373-947d-da1b273b44a8 service glance] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:48:07.395851 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8c9ac-e29d-4938-983e-f35d4625ba8c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd642e290e44e67b56eb3b8361d7af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:48:07.396542 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8c9ac-e29d-4938-983e-f35d4625ba8c service swift] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:07.396833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8c9ac-e29d-4938-983e-f35d4625ba8c service swift] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:07.397135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8c9ac-e29d-4938-983e-f35d4625ba8c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:07.413330 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8c9ac-e29d-4938-983e-f35d4625ba8c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:48:07.415991 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f8c9ac-e29d-4938-983e-f35d4625ba8c service swift] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:48:07.441564 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1202/2388] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jun 22 11:48:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 22 11:48:11.541396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d37e7-1281-4a6a-879c-56a8771bf2fa None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:11.541944 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d37e7-1281-4a6a-879c-56a8771bf2fa None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:11.541944 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d37e7-1281-4a6a-879c-56a8771bf2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:11.575165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-858d37e7-1281-4a6a-879c-56a8771bf2fa None None] MFA Rules not processed for user `955c44ffe1bd426e900c7405df44d1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:48:11.576470 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-858d37e7-1281-4a6a-879c-56a8771bf2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:48:11.586329 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1187/2389] 10.222.0.36 () {58 vars in 967 bytes} [Wed Jun 22 11:48:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:48:11.600989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8c9ac-e29d-4938-983e-f35d4625ba8c service swift] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:48:11.617533 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8c9ac-e29d-4938-983e-f35d4625ba8c service swift] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:48:11.625646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4df1a-ddea-44fa-aa2a-76834915d907 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd642e290e44e67b56eb3b8361d7af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:48:11.627089 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4df1a-ddea-44fa-aa2a-76834915d907 service swift] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:11.627549 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4df1a-ddea-44fa-aa2a-76834915d907 service swift] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:11.627850 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4df1a-ddea-44fa-aa2a-76834915d907 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:11.640467 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4df1a-ddea-44fa-aa2a-76834915d907 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:48:11.642417 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4df1a-ddea-44fa-aa2a-76834915d907 service swift] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:48:11.654200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1203/2390] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jun 22 11:48:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 22 11:48:14.301740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d37e7-1281-4a6a-879c-56a8771bf2fa None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:48:14.303008 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-858d37e7-1281-4a6a-879c-56a8771bf2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:48:14.425444 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d37e7-1281-4a6a-879c-56a8771bf2fa None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:48:14.431326 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87b382-4de8-4a1b-aba5-70885c7efed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:48:14.432245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87b382-4de8-4a1b-aba5-70885c7efed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:14.432537 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87b382-4de8-4a1b-aba5-70885c7efed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:14.432812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87b382-4de8-4a1b-aba5-70885c7efed0 admin admin] PATH_INFO: `/v3/users/72365998f1bb459cbff28ca0b267fe69` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:14.434114 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87b382-4de8-4a1b-aba5-70885c7efed0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72365998f1bb459cbff28ca0b267fe69)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:48:14.437040 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87b382-4de8-4a1b-aba5-70885c7efed0 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:48:14.504691 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cd87b382-4de8-4a1b-aba5-70885c7efed0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72365998f1bb459cbff28ca0b267fe69'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:48:14.519799 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cd87b382-4de8-4a1b-aba5-70885c7efed0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72365998f1bb459cbff28ca0b267fe69'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:48:14.531213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cd87b382-4de8-4a1b-aba5-70885c7efed0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72365998f1bb459cbff28ca0b267fe69'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:48:14.532072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cd87b382-4de8-4a1b-aba5-70885c7efed0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72365998f1bb459cbff28ca0b267fe69'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:48:14.538468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:48:14.540624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cd87b382-4de8-4a1b-aba5-70885c7efed0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72365998f1bb459cbff28ca0b267fe69'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:48:14.552996 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1188/2391] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:48:14 2022] DELETE /identity/v3/users/72365998f1bb459cbff28ca0b267fe69 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:48:14.758824 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4df1a-ddea-44fa-aa2a-76834915d907 service swift] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:48:14.773459 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4df1a-ddea-44fa-aa2a-76834915d907 service swift] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:48:14.811407 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:48:14.848423 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60352bf0-1dcb-40d3-9692-9a094364b5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:48:14.849566 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60352bf0-1dcb-40d3-9692-9a094364b5bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:14.850050 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60352bf0-1dcb-40d3-9692-9a094364b5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:14.850327 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60352bf0-1dcb-40d3-9692-9a094364b5bb admin admin] PATH_INFO: `/v3/projects/97d7e536c4254f469c6da669481eb8e6` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:14.851609 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60352bf0-1dcb-40d3-9692-9a094364b5bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=97d7e536c4254f469c6da669481eb8e6)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:48:14.854251 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60352bf0-1dcb-40d3-9692-9a094364b5bb admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:48:14.916772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-60352bf0-1dcb-40d3-9692-9a094364b5bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97d7e536c4254f469c6da669481eb8e6', 'request_id': 'req-60352bf0-1dcb-40d3-9692-9a094364b5bb'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:48:14.916772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-60352bf0-1dcb-40d3-9692-9a094364b5bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97d7e536c4254f469c6da669481eb8e6', 'request_id': 'req-60352bf0-1dcb-40d3-9692-9a094364b5bb'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:48:14.927539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-60352bf0-1dcb-40d3-9692-9a094364b5bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97d7e536c4254f469c6da669481eb8e6', 'request_id': 'req-60352bf0-1dcb-40d3-9692-9a094364b5bb'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:48:14.942145 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-60352bf0-1dcb-40d3-9692-9a094364b5bb admin admin] The token cache is being invalidate because project 97d7e536c4254f469c6da669481eb8e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:48:14.942496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-60352bf0-1dcb-40d3-9692-9a094364b5bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:48:14.943192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-60352bf0-1dcb-40d3-9692-9a094364b5bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:48:14.945338 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1204/2392] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:48:14 2022] DELETE /identity/v3/projects/97d7e536c4254f469c6da669481eb8e6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:48:14.949932 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac749a87-d45b-4c17-ab0c-4bca88fb57e3 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:14.950352 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac749a87-d45b-4c17-ab0c-4bca88fb57e3 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:14.950743 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac749a87-d45b-4c17-ab0c-4bca88fb57e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:14.991245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-ac749a87-d45b-4c17-ab0c-4bca88fb57e3 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:48:14.992505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ac749a87-d45b-4c17-ab0c-4bca88fb57e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:48:15.069822 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1189/2393] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:48:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:48:15.078635 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316543a-d660-4cd6-806c-74b74719afa5 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:15.079056 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316543a-d660-4cd6-806c-74b74719afa5 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:15.079440 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316543a-d660-4cd6-806c-74b74719afa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:15.105204 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-1316543a-d660-4cd6-806c-74b74719afa5 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:48:15.109316 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-1316543a-d660-4cd6-806c-74b74719afa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:48:15.139526 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1205/2394] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:48:15.144570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac749a87-d45b-4c17-ab0c-4bca88fb57e3 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:48:15.155887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac749a87-d45b-4c17-ab0c-4bca88fb57e3 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:48:15.161202 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142ab78-fff5-48f6-8ce8-be9dc5a607c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:48:15.161957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142ab78-fff5-48f6-8ce8-be9dc5a607c6 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:15.162232 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142ab78-fff5-48f6-8ce8-be9dc5a607c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:15.162512 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142ab78-fff5-48f6-8ce8-be9dc5a607c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:15.163246 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142ab78-fff5-48f6-8ce8-be9dc5a607c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:48:15.163835 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142ab78-fff5-48f6-8ce8-be9dc5a607c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:48:15.165205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142ab78-fff5-48f6-8ce8-be9dc5a607c6 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:48:15.178490 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1190/2395] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:48:15.183596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316543a-d660-4cd6-806c-74b74719afa5 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:48:15.194000 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316543a-d660-4cd6-806c-74b74719afa5 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:48:15.198917 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225f8c1-9a74-4746-9603-f7adf9bd09e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:48:15.200134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225f8c1-9a74-4746-9603-f7adf9bd09e4 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:15.200431 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225f8c1-9a74-4746-9603-f7adf9bd09e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:15.201157 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225f8c1-9a74-4746-9603-f7adf9bd09e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:15.201397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225f8c1-9a74-4746-9603-f7adf9bd09e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:48:15.203342 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225f8c1-9a74-4746-9603-f7adf9bd09e4 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:48:15.236454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1206/2396] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:48:15 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:48:15.240250 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142ab78-fff5-48f6-8ce8-be9dc5a607c6 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:48:15.251549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142ab78-fff5-48f6-8ce8-be9dc5a607c6 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:48:15.329162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881440c8-860e-402b-8d7a-2a603bff71f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:48:15.330763 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881440c8-860e-402b-8d7a-2a603bff71f9 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:15.331170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881440c8-860e-402b-8d7a-2a603bff71f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:15.331685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881440c8-860e-402b-8d7a-2a603bff71f9 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:15.332639 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881440c8-860e-402b-8d7a-2a603bff71f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:48:15.334965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881440c8-860e-402b-8d7a-2a603bff71f9 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:48:15.369205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1191/2397] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:48:15 2022] POST /identity/v3/users => generated 285 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:48:15.370709 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225f8c1-9a74-4746-9603-f7adf9bd09e4 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:48:15.387002 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225f8c1-9a74-4746-9603-f7adf9bd09e4 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:48:15.393043 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e8828-2412-4b63-82aa-87ae0154d45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:48:15.393796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e8828-2412-4b63-82aa-87ae0154d45b admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:15.394138 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e8828-2412-4b63-82aa-87ae0154d45b admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:15.394530 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e8828-2412-4b63-82aa-87ae0154d45b admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:15.395719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e8828-2412-4b63-82aa-87ae0154d45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:48:15.397816 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e8828-2412-4b63-82aa-87ae0154d45b admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:48:15.416667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1207/2398] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:48:15.420106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881440c8-860e-402b-8d7a-2a603bff71f9 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:48:15.431642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881440c8-860e-402b-8d7a-2a603bff71f9 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:48:15.438850 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d26a4-4d73-4742-be8a-b8d8b23d37c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:48:15.439665 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d26a4-4d73-4742-be8a-b8d8b23d37c8 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:15.440042 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d26a4-4d73-4742-be8a-b8d8b23d37c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:15.440393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d26a4-4d73-4742-be8a-b8d8b23d37c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:15.441201 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d26a4-4d73-4742-be8a-b8d8b23d37c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:48:15.442824 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d26a4-4d73-4742-be8a-b8d8b23d37c8 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:48:15.456091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1192/2399] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:48:15.459953 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e8828-2412-4b63-82aa-87ae0154d45b admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:48:15.472373 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e8828-2412-4b63-82aa-87ae0154d45b admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:48:15.478145 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872402f7-a9d6-4e66-b3fa-df8a39d2e927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:48:15.479151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872402f7-a9d6-4e66-b3fa-df8a39d2e927 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:15.481798 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872402f7-a9d6-4e66-b3fa-df8a39d2e927 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:15.482197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872402f7-a9d6-4e66-b3fa-df8a39d2e927 admin admin] PATH_INFO: `/v3/projects/6961af8bbea04165bfb4c8a90b713d2c/users/dfc0aa753e7a4e7691a72b624fc3fe81/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:15.505978 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872402f7-a9d6-4e66-b3fa-df8a39d2e927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6961af8bbea04165bfb4c8a90b713d2c, user_id=dfc0aa753e7a4e7691a72b624fc3fe81, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:48:15.509243 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872402f7-a9d6-4e66-b3fa-df8a39d2e927 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:48:15.531643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1208/2400] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:48:15 2022] PUT /identity/v3/projects/6961af8bbea04165bfb4c8a90b713d2c/users/dfc0aa753e7a4e7691a72b624fc3fe81/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:48:15.542519 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542b162-514f-4520-85bd-13b6f1a28545 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:15.542887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542b162-514f-4520-85bd-13b6f1a28545 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:15.543231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542b162-514f-4520-85bd-13b6f1a28545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:15.600235 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-5542b162-514f-4520-85bd-13b6f1a28545 None None] MFA Rules not processed for user `dfc0aa753e7a4e7691a72b624fc3fe81`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:48:15.601461 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-5542b162-514f-4520-85bd-13b6f1a28545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:48:15.671628 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:48:15.703399 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1193/2401] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:48:15 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:48:15.714149 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872402f7-a9d6-4e66-b3fa-df8a39d2e927 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:48:15.715941 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-872402f7-a9d6-4e66-b3fa-df8a39d2e927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:48:15.787766 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872402f7-a9d6-4e66-b3fa-df8a39d2e927 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:48:15.792266 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcd33d-8043-4ea9-8923-ed5044c0177e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ed133721ed044708ee362c93f564d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:48:15.796002 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcd33d-8043-4ea9-8923-ed5044c0177e service glance] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:15.796384 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcd33d-8043-4ea9-8923-ed5044c0177e service glance] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:15.796783 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcd33d-8043-4ea9-8923-ed5044c0177e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:15.807305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adcd33d-8043-4ea9-8923-ed5044c0177e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:48:15.809586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adcd33d-8043-4ea9-8923-ed5044c0177e service glance] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:48:15.817657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:48:15.822998 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1209/2402] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:48:15 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:48:17.874594 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38632ccf-a1f6-49d1-8ff0-125679ee24b7 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:17.875265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38632ccf-a1f6-49d1-8ff0-125679ee24b7 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:17.875265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38632ccf-a1f6-49d1-8ff0-125679ee24b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:17.893231 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-38632ccf-a1f6-49d1-8ff0-125679ee24b7 None None] MFA Rules not processed for user `8fc6cd29af4d4c28bec9bf6cc9db5749`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:48:17.894428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-38632ccf-a1f6-49d1-8ff0-125679ee24b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:48:17.957134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1194/2403] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:48:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:48:19.174708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcd33d-8043-4ea9-8923-ed5044c0177e service glance] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:48:19.176285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5adcd33d-8043-4ea9-8923-ed5044c0177e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:48:19.284838 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcd33d-8043-4ea9-8923-ed5044c0177e service glance] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:48:19.294652 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9220f25-6ccc-448c-bc2c-7432eb204e6a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93630ed5e6dc4a5f958bc7cf1a68a26c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:48:19.295322 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9220f25-6ccc-448c-bc2c-7432eb204e6a service placement] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:19.295634 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9220f25-6ccc-448c-bc2c-7432eb204e6a service placement] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:19.295915 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9220f25-6ccc-448c-bc2c-7432eb204e6a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:19.297738 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-a9220f25-6ccc-448c-bc2c-7432eb204e6a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:48:19.310864 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9220f25-6ccc-448c-bc2c-7432eb204e6a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:48:19.312498 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9220f25-6ccc-448c-bc2c-7432eb204e6a service placement] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:48:19.323201 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1210/2404] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:48:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:48:20.522761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51358462-259c-4aa6-bec2-be98e988e034 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:20.523268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51358462-259c-4aa6-bec2-be98e988e034 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:20.523385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51358462-259c-4aa6-bec2-be98e988e034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:20.555636 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-51358462-259c-4aa6-bec2-be98e988e034 None None] MFA Rules not processed for user `955c44ffe1bd426e900c7405df44d1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:48:20.556906 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-51358462-259c-4aa6-bec2-be98e988e034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:48:20.629281 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1195/2405] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jun 22 11:48:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:48:20.643135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9220f25-6ccc-448c-bc2c-7432eb204e6a service placement] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:48:20.645270 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-a9220f25-6ccc-448c-bc2c-7432eb204e6a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:48:20.750475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9220f25-6ccc-448c-bc2c-7432eb204e6a service placement] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:48:20.760410 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea363c-74c5-4ad1-8d2f-f14cd61cd5a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd642e290e44e67b56eb3b8361d7af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:48:20.761194 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea363c-74c5-4ad1-8d2f-f14cd61cd5a5 service swift] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:48:20.761583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea363c-74c5-4ad1-8d2f-f14cd61cd5a5 service swift] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:48:20.762053 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea363c-74c5-4ad1-8d2f-f14cd61cd5a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:48:20.774211 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea363c-74c5-4ad1-8d2f-f14cd61cd5a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:48:20.776175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea363c-74c5-4ad1-8d2f-f14cd61cd5a5 service swift] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:48:20.787300 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1211/2406] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jun 22 11:48:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 22 11:48:30.620880 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=58 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:48:30.787596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=73 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:49:14.413273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51358462-259c-4aa6-bec2-be98e988e034 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:49:14.414743 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-51358462-259c-4aa6-bec2-be98e988e034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:49:14.470548 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51358462-259c-4aa6-bec2-be98e988e034 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:49:14.474324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbcc98-7599-4a14-ba32-cbd5582902e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:49:14.475094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbcc98-7599-4a14-ba32-cbd5582902e6 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:49:14.475409 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbcc98-7599-4a14-ba32-cbd5582902e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:49:14.475692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbcc98-7599-4a14-ba32-cbd5582902e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:49:14.477663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ddfbcc98-7599-4a14-ba32-cbd5582902e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:49:14.540128 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbcc98-7599-4a14-ba32-cbd5582902e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:49:14.542213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbcc98-7599-4a14-ba32-cbd5582902e6 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:49:14.551694 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1196/2407] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:49:14 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:49:14.691917 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea363c-74c5-4ad1-8d2f-f14cd61cd5a5 service swift] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:49:14.761812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea363c-74c5-4ad1-8d2f-f14cd61cd5a5 service swift] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:49:14.768303 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cdef2e-a98b-4f52-b6ee-fb37b37792d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:49:14.769185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdef2e-a98b-4f52-b6ee-fb37b37792d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:49:14.769476 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdef2e-a98b-4f52-b6ee-fb37b37792d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:49:14.769748 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdef2e-a98b-4f52-b6ee-fb37b37792d4 admin admin] PATH_INFO: `/v3/users/dfc0aa753e7a4e7691a72b624fc3fe81` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:49:14.770984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cdef2e-a98b-4f52-b6ee-fb37b37792d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfc0aa753e7a4e7691a72b624fc3fe81)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:49:14.772952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cdef2e-a98b-4f52-b6ee-fb37b37792d4 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:49:14.832178 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b0cdef2e-a98b-4f52-b6ee-fb37b37792d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfc0aa753e7a4e7691a72b624fc3fe81'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:49:14.846515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b0cdef2e-a98b-4f52-b6ee-fb37b37792d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfc0aa753e7a4e7691a72b624fc3fe81'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:49:14.855725 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b0cdef2e-a98b-4f52-b6ee-fb37b37792d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfc0aa753e7a4e7691a72b624fc3fe81'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:49:14.871463 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b0cdef2e-a98b-4f52-b6ee-fb37b37792d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfc0aa753e7a4e7691a72b624fc3fe81'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:49:14.872166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b0cdef2e-a98b-4f52-b6ee-fb37b37792d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfc0aa753e7a4e7691a72b624fc3fe81'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:49:14.878358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1212/2408] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:49:14 2022] DELETE /identity/v3/users/dfc0aa753e7a4e7691a72b624fc3fe81 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:49:14.882874 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c713c0-8819-4bc6-8299-87e517131206 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:49:14.883170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c713c0-8819-4bc6-8299-87e517131206 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:49:14.883475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c713c0-8819-4bc6-8299-87e517131206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:49:14.902776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-d6c713c0-8819-4bc6-8299-87e517131206 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:49:14.904604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d6c713c0-8819-4bc6-8299-87e517131206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:49:14.981730 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1197/2409] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:49:14.996678 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cdef2e-a98b-4f52-b6ee-fb37b37792d4 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:49:14.998093 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b0cdef2e-a98b-4f52-b6ee-fb37b37792d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:49:15.074022 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cdef2e-a98b-4f52-b6ee-fb37b37792d4 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:49:15.078380 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca37b2-061d-4862-b485-07b70bc3384e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:49:15.079055 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca37b2-061d-4862-b485-07b70bc3384e service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:49:15.079318 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca37b2-061d-4862-b485-07b70bc3384e service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:49:15.079633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca37b2-061d-4862-b485-07b70bc3384e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:49:15.095304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca37b2-061d-4862-b485-07b70bc3384e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:49:15.096794 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca37b2-061d-4862-b485-07b70bc3384e service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:49:15.106861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1213/2410] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:49:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:49:15.406833 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c713c0-8819-4bc6-8299-87e517131206 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:49:15.408428 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d6c713c0-8819-4bc6-8299-87e517131206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:49:15.433134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c713c0-8819-4bc6-8299-87e517131206 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:49:15.441826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbd801-d91f-48a4-a825-33d7a815e73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:49:15.443394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbd801-d91f-48a4-a825-33d7a815e73e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:49:15.444089 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbd801-d91f-48a4-a825-33d7a815e73e admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:49:15.444857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbbd801-d91f-48a4-a825-33d7a815e73e admin admin] PATH_INFO: `/v3/projects/6961af8bbea04165bfb4c8a90b713d2c` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:49:15.446343 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbbd801-d91f-48a4-a825-33d7a815e73e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6961af8bbea04165bfb4c8a90b713d2c)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:49:15.448773 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbbd801-d91f-48a4-a825-33d7a815e73e admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:49:15.497499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:49:15.500418 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-acbbd801-d91f-48a4-a825-33d7a815e73e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6961af8bbea04165bfb4c8a90b713d2c', 'request_id': 'req-acbbd801-d91f-48a4-a825-33d7a815e73e'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:49:15.509619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-acbbd801-d91f-48a4-a825-33d7a815e73e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6961af8bbea04165bfb4c8a90b713d2c', 'request_id': 'req-acbbd801-d91f-48a4-a825-33d7a815e73e'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:49:15.517303 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-acbbd801-d91f-48a4-a825-33d7a815e73e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6961af8bbea04165bfb4c8a90b713d2c', 'request_id': 'req-acbbd801-d91f-48a4-a825-33d7a815e73e'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:49:15.528536 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-acbbd801-d91f-48a4-a825-33d7a815e73e admin admin] The token cache is being invalidate because project 6961af8bbea04165bfb4c8a90b713d2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:49:15.528828 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-acbbd801-d91f-48a4-a825-33d7a815e73e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:49:15.529647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-acbbd801-d91f-48a4-a825-33d7a815e73e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:49:15.531509 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1198/2411] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:49:15 2022] DELETE /identity/v3/projects/6961af8bbea04165bfb4c8a90b713d2c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:49:17.299657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca37b2-061d-4862-b485-07b70bc3384e service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:49:17.312411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca37b2-061d-4862-b485-07b70bc3384e service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:49:17.348216 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:49:17.416850 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6610fd-a4f5-437f-bb17-8d15a2495cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:49:17.417939 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6610fd-a4f5-437f-bb17-8d15a2495cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:49:17.418443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6610fd-a4f5-437f-bb17-8d15a2495cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:49:17.418952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6610fd-a4f5-437f-bb17-8d15a2495cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:49:17.421172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-4d6610fd-a4f5-437f-bb17-8d15a2495cc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:49:17.520192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6610fd-a4f5-437f-bb17-8d15a2495cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:49:17.521682 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6610fd-a4f5-437f-bb17-8d15a2495cc1 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:49:17.533513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1214/2412] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:49:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:49:25.525542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:49:27.528729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=40 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:50:09.913517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbd801-d91f-48a4-a825-33d7a815e73e admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:50:09.914925 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-acbbd801-d91f-48a4-a825-33d7a815e73e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:50:09.927033 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbbd801-d91f-48a4-a825-33d7a815e73e admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:50:09.930606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c5916-4285-4aa3-a460-1a7269c8d9a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:50:09.931378 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c5916-4285-4aa3-a460-1a7269c8d9a1 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:50:09.931570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c5916-4285-4aa3-a460-1a7269c8d9a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:50:09.931847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c5916-4285-4aa3-a460-1a7269c8d9a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:50:09.933585 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a72c5916-4285-4aa3-a460-1a7269c8d9a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:50:10.014391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c5916-4285-4aa3-a460-1a7269c8d9a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:50:10.016115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c5916-4285-4aa3-a460-1a7269c8d9a1 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:50:10.037617 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1199/2413] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:50:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:50:11.207272 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6610fd-a4f5-437f-bb17-8d15a2495cc1 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:50:11.216035 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6610fd-a4f5-437f-bb17-8d15a2495cc1 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:50:11.219675 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c3eb6-5d8d-4162-99ab-3ce19dfa02f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:50:11.220312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c3eb6-5d8d-4162-99ab-3ce19dfa02f9 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:50:11.220594 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c3eb6-5d8d-4162-99ab-3ce19dfa02f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:50:11.220870 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c3eb6-5d8d-4162-99ab-3ce19dfa02f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:50:11.222706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-8a9c3eb6-5d8d-4162-99ab-3ce19dfa02f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:50:11.237350 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c3eb6-5d8d-4162-99ab-3ce19dfa02f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:50:11.238766 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c3eb6-5d8d-4162-99ab-3ce19dfa02f9 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:50:11.250252 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1215/2414] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:50:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:50:20.034475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=43 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:50:21.245452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=6 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:50:34.065268 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afabc7b-a222-4d05-aeeb-f242973fe1d5 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:50:34.065793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afabc7b-a222-4d05-aeeb-f242973fe1d5 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:50:34.065898 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afabc7b-a222-4d05-aeeb-f242973fe1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:50:34.113626 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-0afabc7b-a222-4d05-aeeb-f242973fe1d5 None None] MFA Rules not processed for user `955c44ffe1bd426e900c7405df44d1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:50:34.114825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-0afabc7b-a222-4d05-aeeb-f242973fe1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:50:34.184449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1200/2415] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jun 22 11:50:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:50:34.223127 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c3eb6-5d8d-4162-99ab-3ce19dfa02f9 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:50:34.224591 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-8a9c3eb6-5d8d-4162-99ab-3ce19dfa02f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:50:34.319087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c3eb6-5d8d-4162-99ab-3ce19dfa02f9 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:50:34.324107 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc9a39-9291-4104-8ef1-a1e777759657 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd642e290e44e67b56eb3b8361d7af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:50:34.324744 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc9a39-9291-4104-8ef1-a1e777759657 service swift] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:50:34.325027 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc9a39-9291-4104-8ef1-a1e777759657 service swift] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:50:34.325304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc9a39-9291-4104-8ef1-a1e777759657 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:50:34.333641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc9a39-9291-4104-8ef1-a1e777759657 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:50:34.335614 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc9a39-9291-4104-8ef1-a1e777759657 service swift] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:50:34.343789 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1216/2416] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jun 22 11:50:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 22 11:50:44.181345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=40 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:50:44.340564 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=41 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:50:51.777627 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afabc7b-a222-4d05-aeeb-f242973fe1d5 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:50:51.778936 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-0afabc7b-a222-4d05-aeeb-f242973fe1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:50:51.794877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afabc7b-a222-4d05-aeeb-f242973fe1d5 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:50:51.800315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ec5d8-bd5f-4e3f-901e-32eb07bcc5fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:50:51.800948 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ec5d8-bd5f-4e3f-901e-32eb07bcc5fe service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:50:51.801227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ec5d8-bd5f-4e3f-901e-32eb07bcc5fe service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:50:51.801499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ec5d8-bd5f-4e3f-901e-32eb07bcc5fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:50:51.803218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-bf4ec5d8-bd5f-4e3f-901e-32eb07bcc5fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:50:51.861542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ec5d8-bd5f-4e3f-901e-32eb07bcc5fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:50:51.863162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ec5d8-bd5f-4e3f-901e-32eb07bcc5fe service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:50:51.872812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1201/2417] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:50:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:50:52.246693 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc9a39-9291-4104-8ef1-a1e777759657 service swift] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:50:52.248170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-adcc9a39-9291-4104-8ef1-a1e777759657 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:50:52.261247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc9a39-9291-4104-8ef1-a1e777759657 service swift] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:50:52.265485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672551fd-92db-4e9a-953d-48a82b9deab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:50:52.266122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672551fd-92db-4e9a-953d-48a82b9deab1 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:50:52.266402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672551fd-92db-4e9a-953d-48a82b9deab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:50:52.266680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672551fd-92db-4e9a-953d-48a82b9deab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:50:52.268477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-672551fd-92db-4e9a-953d-48a82b9deab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:50:52.279320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672551fd-92db-4e9a-953d-48a82b9deab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:50:52.280879 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672551fd-92db-4e9a-953d-48a82b9deab1 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:50:52.289659 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1217/2418] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:50:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:50:54.913138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ec5d8-bd5f-4e3f-901e-32eb07bcc5fe service nova] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:50:54.915049 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-bf4ec5d8-bd5f-4e3f-901e-32eb07bcc5fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:50:54.930601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ec5d8-bd5f-4e3f-901e-32eb07bcc5fe service nova] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:50:54.936360 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81eb2f-2598-48e2-bf85-6c1acde206cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:50:54.937011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81eb2f-2598-48e2-bf85-6c1acde206cb service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:50:54.937296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81eb2f-2598-48e2-bf85-6c1acde206cb service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:50:54.942393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81eb2f-2598-48e2-bf85-6c1acde206cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:50:54.942393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-9c81eb2f-2598-48e2-bf85-6c1acde206cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:50:54.952887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81eb2f-2598-48e2-bf85-6c1acde206cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:50:54.954730 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81eb2f-2598-48e2-bf85-6c1acde206cb service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:50:54.966397 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1202/2419] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:50:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:50:55.657571 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672551fd-92db-4e9a-953d-48a82b9deab1 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:50:55.667162 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672551fd-92db-4e9a-953d-48a82b9deab1 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:50:55.671244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405f69c-3f7f-4127-9dce-de664d2061aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:50:55.671921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405f69c-3f7f-4127-9dce-de664d2061aa service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:50:55.672205 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405f69c-3f7f-4127-9dce-de664d2061aa service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:50:55.672502 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405f69c-3f7f-4127-9dce-de664d2061aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:50:55.674326 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-7405f69c-3f7f-4127-9dce-de664d2061aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:50:55.684181 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405f69c-3f7f-4127-9dce-de664d2061aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:50:55.685617 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405f69c-3f7f-4127-9dce-de664d2061aa service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:50:55.693577 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1218/2420] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:50:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:50:55.800259 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1573d-2c23-46ca-8d87-e74396a4e5fd None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:50:55.800600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1573d-2c23-46ca-8d87-e74396a4e5fd None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:50:55.800891 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1573d-2c23-46ca-8d87-e74396a4e5fd None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:50:55.802282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1203/2421] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jun 22 11:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:50:55.808791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a99fc-ec6d-40a5-86b6-86b3d7a6d54b None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:50:55.809193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a99fc-ec6d-40a5-86b6-86b3d7a6d54b None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:50:55.809467 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a99fc-ec6d-40a5-86b6-86b3d7a6d54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:50:55.849311 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-596a99fc-ec6d-40a5-86b6-86b3d7a6d54b None None] MFA Rules not processed for user `479a376dcac14c509c1c55f0046d88cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:50:55.851103 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-596a99fc-ec6d-40a5-86b6-86b3d7a6d54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:50:55.860800 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1219/2422] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jun 22 11:50:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:50:55.873801 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1573d-2c23-46ca-8d87-e74396a4e5fd None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:50:55.875541 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f9a1573d-2c23-46ca-8d87-e74396a4e5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:50:55.975270 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1573d-2c23-46ca-8d87-e74396a4e5fd None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:50:55.978955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04219f5-e013-443d-a880-72941f0aaf44 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ed133721ed044708ee362c93f564d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:50:55.979672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04219f5-e013-443d-a880-72941f0aaf44 service glance] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:50:55.979975 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04219f5-e013-443d-a880-72941f0aaf44 service glance] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:50:55.980260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04219f5-e013-443d-a880-72941f0aaf44 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:50:55.990295 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04219f5-e013-443d-a880-72941f0aaf44 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:50:55.991826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04219f5-e013-443d-a880-72941f0aaf44 service glance] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:50:56.002396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1204/2423] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:50:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:50:56.068650 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8814ef8-90a5-4e1a-83a8-e339a8b84048 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:50:56.068935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8814ef8-90a5-4e1a-83a8-e339a8b84048 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:50:56.069724 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8814ef8-90a5-4e1a-83a8-e339a8b84048 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:50:56.071119 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1220/2424] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jun 22 11:50:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:50:56.075829 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a13c0-2f8c-4eae-9a0d-0d3e31713717 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:50:56.076222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a13c0-2f8c-4eae-9a0d-0d3e31713717 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:50:56.076399 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a13c0-2f8c-4eae-9a0d-0d3e31713717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:50:56.095178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-fa1a13c0-2f8c-4eae-9a0d-0d3e31713717 None None] MFA Rules not processed for user `c00d0bb777a044a3a4d4540d187d938a`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:50:56.096397 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-fa1a13c0-2f8c-4eae-9a0d-0d3e31713717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:50:56.102834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1205/2425] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jun 22 11:50:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:50:56.114348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8814ef8-90a5-4e1a-83a8-e339a8b84048 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:50:56.125327 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8814ef8-90a5-4e1a-83a8-e339a8b84048 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:50:56.128784 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145fa79-08ff-4a76-af94-b4f90df780ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:50:56.129413 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145fa79-08ff-4a76-af94-b4f90df780ed service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:50:56.129701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145fa79-08ff-4a76-af94-b4f90df780ed service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:50:56.129977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145fa79-08ff-4a76-af94-b4f90df780ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:50:56.137672 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145fa79-08ff-4a76-af94-b4f90df780ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:50:56.139093 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145fa79-08ff-4a76-af94-b4f90df780ed service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:50:56.147063 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1221/2426] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:50:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:50:56.234389 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686db68b-7e96-409c-9820-ac57076bde06 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:50:56.234674 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686db68b-7e96-409c-9820-ac57076bde06 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:50:56.234950 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686db68b-7e96-409c-9820-ac57076bde06 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:50:56.236489 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1206/2427] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jun 22 11:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:50:56.242308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a6bee-6d0c-49b8-bd8f-34af3738daa1 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:50:56.242592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a6bee-6d0c-49b8-bd8f-34af3738daa1 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:50:56.242867 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a6bee-6d0c-49b8-bd8f-34af3738daa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:50:56.261348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f51a6bee-6d0c-49b8-bd8f-34af3738daa1 None None] MFA Rules not processed for user `479a376dcac14c509c1c55f0046d88cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:50:56.262539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f51a6bee-6d0c-49b8-bd8f-34af3738daa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:50:56.272071 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1222/2428] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jun 22 11:50:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:50:56.283056 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686db68b-7e96-409c-9820-ac57076bde06 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:50:56.294069 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686db68b-7e96-409c-9820-ac57076bde06 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:50:56.297566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d477c9b-1f63-4d64-bf9e-c7ec4c8222b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:50:56.298188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d477c9b-1f63-4d64-bf9e-c7ec4c8222b6 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:50:56.298469 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d477c9b-1f63-4d64-bf9e-c7ec4c8222b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:50:56.298745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d477c9b-1f63-4d64-bf9e-c7ec4c8222b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:50:56.309513 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d477c9b-1f63-4d64-bf9e-c7ec4c8222b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:50:56.310929 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d477c9b-1f63-4d64-bf9e-c7ec4c8222b6 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:50:56.322017 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1207/2429] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:50:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:50:56.384056 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a6bee-6d0c-49b8-bd8f-34af3738daa1 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:50:56.385468 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f51a6bee-6d0c-49b8-bd8f-34af3738daa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:50:56.455276 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a6bee-6d0c-49b8-bd8f-34af3738daa1 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:50:56.460483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00f8cc-bc27-4a4b-8dad-edf279de50fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93630ed5e6dc4a5f958bc7cf1a68a26c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:50:56.461135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00f8cc-bc27-4a4b-8dad-edf279de50fd service placement] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:50:56.461420 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00f8cc-bc27-4a4b-8dad-edf279de50fd service placement] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:50:56.461700 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c00f8cc-bc27-4a4b-8dad-edf279de50fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:50:56.463563 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-8c00f8cc-bc27-4a4b-8dad-edf279de50fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:50:56.475226 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c00f8cc-bc27-4a4b-8dad-edf279de50fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:50:56.476865 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c00f8cc-bc27-4a4b-8dad-edf279de50fd service placement] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:50:56.486582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1223/2430] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:50:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:51:03.581535 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8de0f3-ced6-4412-a10d-438d63b97f44 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:51:03.582041 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8de0f3-ced6-4412-a10d-438d63b97f44 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:51:03.582133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8de0f3-ced6-4412-a10d-438d63b97f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:51:03.599267 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-0e8de0f3-ced6-4412-a10d-438d63b97f44 None None] MFA Rules not processed for user `955c44ffe1bd426e900c7405df44d1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:51:03.600988 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-0e8de0f3-ced6-4412-a10d-438d63b97f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:51:03.608162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1208/2431] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jun 22 11:51:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:51:03.620596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00f8cc-bc27-4a4b-8dad-edf279de50fd service placement] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:51:03.631694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c00f8cc-bc27-4a4b-8dad-edf279de50fd service placement] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:51:03.636647 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd0c388-564f-460b-a46f-1c589fb7a317 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd642e290e44e67b56eb3b8361d7af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:51:03.637518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0c388-564f-460b-a46f-1c589fb7a317 service swift] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:51:03.637806 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0c388-564f-460b-a46f-1c589fb7a317 service swift] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:51:03.638085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0c388-564f-460b-a46f-1c589fb7a317 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:51:03.646559 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd0c388-564f-460b-a46f-1c589fb7a317 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:51:03.648154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd0c388-564f-460b-a46f-1c589fb7a317 service swift] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:51:03.656253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1224/2432] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jun 22 11:51:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 22 11:51:08.067618 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8de0f3-ced6-4412-a10d-438d63b97f44 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:51:08.080995 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8de0f3-ced6-4412-a10d-438d63b97f44 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:51:08.086527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4690030-fd11-4f72-924d-0c97c7653d2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:51:08.089092 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4690030-fd11-4f72-924d-0c97c7653d2e service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:51:08.091911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4690030-fd11-4f72-924d-0c97c7653d2e service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:51:08.092219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4690030-fd11-4f72-924d-0c97c7653d2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:51:08.094132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d4690030-fd11-4f72-924d-0c97c7653d2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:51:08.138805 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:51:08.158092 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4690030-fd11-4f72-924d-0c97c7653d2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:51:08.159689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4690030-fd11-4f72-924d-0c97c7653d2e service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:51:08.169891 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1209/2433] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:51:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:51:13.652953 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=68 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:51:18.165448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=9 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:51:29.860364 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c88fac-ad70-413e-9bd7-f4317218d661 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:51:29.860966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c88fac-ad70-413e-9bd7-f4317218d661 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:51:29.860966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c88fac-ad70-413e-9bd7-f4317218d661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:51:29.878815 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-51c88fac-ad70-413e-9bd7-f4317218d661 None None] MFA Rules not processed for user `27cc3d38926f470c91eeb6aecfc86a48`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:51:29.880076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-51c88fac-ad70-413e-9bd7-f4317218d661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:51:29.887159 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1225/2434] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:51:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:51:38.863340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff78c2-ee83-485c-a014-c845f5ff4987 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:51:38.863889 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff78c2-ee83-485c-a014-c845f5ff4987 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:51:38.863981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff78c2-ee83-485c-a014-c845f5ff4987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:51:38.885914 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-bdff78c2-ee83-485c-a014-c845f5ff4987 None None] MFA Rules not processed for user `955c44ffe1bd426e900c7405df44d1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:51:38.887520 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-bdff78c2-ee83-485c-a014-c845f5ff4987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:51:38.896909 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1210/2435] 10.222.0.36 () {58 vars in 967 bytes} [Wed Jun 22 11:51:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:51:38.920032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c88fac-ad70-413e-9bd7-f4317218d661 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:51:38.936475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c88fac-ad70-413e-9bd7-f4317218d661 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:51:38.941660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e57b2-f495-4cb5-9401-f306b5fac577 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd642e290e44e67b56eb3b8361d7af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:51:38.942312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e57b2-f495-4cb5-9401-f306b5fac577 service swift] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:51:38.942759 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e57b2-f495-4cb5-9401-f306b5fac577 service swift] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:51:38.943190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e57b2-f495-4cb5-9401-f306b5fac577 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:51:38.952315 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e57b2-f495-4cb5-9401-f306b5fac577 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:51:38.953942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e57b2-f495-4cb5-9401-f306b5fac577 service swift] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:51:38.963609 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1226/2436] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jun 22 11:51:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 22 11:51:39.293272 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff78c2-ee83-485c-a014-c845f5ff4987 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:51:39.307785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff78c2-ee83-485c-a014-c845f5ff4987 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:51:39.314280 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4917ac-34a3-47cc-9163-bf0276e6ab1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:51:39.315302 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4917ac-34a3-47cc-9163-bf0276e6ab1c service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:51:39.315982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4917ac-34a3-47cc-9163-bf0276e6ab1c service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:51:39.316558 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4917ac-34a3-47cc-9163-bf0276e6ab1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:51:39.331645 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-9e4917ac-34a3-47cc-9163-bf0276e6ab1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:51:39.361199 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4917ac-34a3-47cc-9163-bf0276e6ab1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:51:39.363117 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4917ac-34a3-47cc-9163-bf0276e6ab1c service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:51:39.375389 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1211/2437] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:51:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:51:48.959462 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=9 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:51:49.370358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=9 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:52:41.004415 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145eaa1-ebdf-472e-9e93-472a3e018ff4 None None] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:52:41.005019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145eaa1-ebdf-472e-9e93-472a3e018ff4 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:52:41.005019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145eaa1-ebdf-472e-9e93-472a3e018ff4 None None] PATH_INFO: `/` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:52:41.006505 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1227/2438] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jun 22 11:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:52:41.013443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044c95f-00d0-40ab-b4cd-22562d1ade78 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:52:41.014210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044c95f-00d0-40ab-b4cd-22562d1ade78 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:52:41.014365 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044c95f-00d0-40ab-b4cd-22562d1ade78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:52:41.033502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-9044c95f-00d0-40ab-b4cd-22562d1ade78 None None] MFA Rules not processed for user `c00d0bb777a044a3a4d4540d187d938a`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:52:41.034828 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-9044c95f-00d0-40ab-b4cd-22562d1ade78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:52:41.041702 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1212/2439] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jun 22 11:52:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:52:41.055316 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145eaa1-ebdf-472e-9e93-472a3e018ff4 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:52:41.066401 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145eaa1-ebdf-472e-9e93-472a3e018ff4 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:52:41.069988 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1bcb12-f3aa-4acb-bf97-de07355692b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:52:41.070624 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bcb12-f3aa-4acb-bf97-de07355692b2 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:52:41.070907 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bcb12-f3aa-4acb-bf97-de07355692b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:52:41.071185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bcb12-f3aa-4acb-bf97-de07355692b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:52:41.079884 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1bcb12-f3aa-4acb-bf97-de07355692b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:52:41.081308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1bcb12-f3aa-4acb-bf97-de07355692b2 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:52:41.090266 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1228/2440] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:52:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:52:41.173359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b18c6-74e5-4b5e-a8b4-dc546de8e3de None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:52:41.173667 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b18c6-74e5-4b5e-a8b4-dc546de8e3de None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:52:41.173945 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b18c6-74e5-4b5e-a8b4-dc546de8e3de None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:52:41.175315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1213/2441] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jun 22 11:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:52:41.180712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5217cc6-4b60-475f-99e0-89a96a5ac527 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:52:41.181012 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5217cc6-4b60-475f-99e0-89a96a5ac527 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:52:41.181292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5217cc6-4b60-475f-99e0-89a96a5ac527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:52:41.200319 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f5217cc6-4b60-475f-99e0-89a96a5ac527 None None] MFA Rules not processed for user `479a376dcac14c509c1c55f0046d88cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:52:41.201516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f5217cc6-4b60-475f-99e0-89a96a5ac527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:52:41.211119 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1229/2442] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jun 22 11:52:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:52:41.221479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b18c6-74e5-4b5e-a8b4-dc546de8e3de None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:52:41.232541 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b18c6-74e5-4b5e-a8b4-dc546de8e3de None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:52:41.241669 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2f609-46ef-43d8-94ec-278c509f6f74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:52:41.242333 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2f609-46ef-43d8-94ec-278c509f6f74 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:52:41.242606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2f609-46ef-43d8-94ec-278c509f6f74 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:52:41.242906 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2f609-46ef-43d8-94ec-278c509f6f74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:52:41.254226 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b2f609-46ef-43d8-94ec-278c509f6f74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:52:41.255668 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b2f609-46ef-43d8-94ec-278c509f6f74 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:52:41.269653 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1214/2443] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:52:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:52:44.824623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5217cc6-4b60-475f-99e0-89a96a5ac527 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:52:44.835875 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5217cc6-4b60-475f-99e0-89a96a5ac527 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:52:44.841372 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61ec4d-714c-4953-b108-5cb8da4b8c4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:52:44.842003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61ec4d-714c-4953-b108-5cb8da4b8c4b service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:52:44.842281 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61ec4d-714c-4953-b108-5cb8da4b8c4b service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:52:44.842555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61ec4d-714c-4953-b108-5cb8da4b8c4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:52:44.844335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-9e61ec4d-714c-4953-b108-5cb8da4b8c4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:52:44.904179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61ec4d-714c-4953-b108-5cb8da4b8c4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:52:44.905801 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61ec4d-714c-4953-b108-5cb8da4b8c4b service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:52:44.914985 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1230/2444] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:52:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:52:51.262944 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=9 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:52:54.910749 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=37 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:10.812000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2f609-46ef-43d8-94ec-278c509f6f74 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:10.823273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2f609-46ef-43d8-94ec-278c509f6f74 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:10.829353 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746cd73-7a68-4525-9f00-2f247c1262b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:10.829982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746cd73-7a68-4525-9f00-2f247c1262b1 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:10.830260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746cd73-7a68-4525-9f00-2f247c1262b1 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:10.830537 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746cd73-7a68-4525-9f00-2f247c1262b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:10.832293 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-4746cd73-7a68-4525-9f00-2f247c1262b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:10.889543 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746cd73-7a68-4525-9f00-2f247c1262b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:10.891100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746cd73-7a68-4525-9f00-2f247c1262b1 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:10.899896 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1215/2445] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:53:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:53:19.082764 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61ec4d-714c-4953-b108-5cb8da4b8c4b service nova] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:19.084266 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-9e61ec4d-714c-4953-b108-5cb8da4b8c4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:19.100348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61ec4d-714c-4953-b108-5cb8da4b8c4b service nova] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:19.106453 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8e2be-9cfa-44d0-914a-f1d7153bbfb9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93630ed5e6dc4a5f958bc7cf1a68a26c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:19.107319 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8e2be-9cfa-44d0-914a-f1d7153bbfb9 service placement] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:19.107791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8e2be-9cfa-44d0-914a-f1d7153bbfb9 service placement] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:19.107791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8e2be-9cfa-44d0-914a-f1d7153bbfb9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:19.109730 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-0cf8e2be-9cfa-44d0-914a-f1d7153bbfb9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:19.122158 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8e2be-9cfa-44d0-914a-f1d7153bbfb9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:19.123787 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8e2be-9cfa-44d0-914a-f1d7153bbfb9 service placement] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:19.135744 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1231/2446] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:53:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:53:19.901312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746cd73-7a68-4525-9f00-2f247c1262b1 service nova] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:19.913197 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746cd73-7a68-4525-9f00-2f247c1262b1 service nova] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:19.919208 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c27781-f884-4d56-b2f7-53102bef36f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:19.919901 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c27781-f884-4d56-b2f7-53102bef36f3 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:19.920213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c27781-f884-4d56-b2f7-53102bef36f3 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:19.920492 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c27781-f884-4d56-b2f7-53102bef36f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:19.922311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-25c27781-f884-4d56-b2f7-53102bef36f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:19.992491 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c27781-f884-4d56-b2f7-53102bef36f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:19.994518 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c27781-f884-4d56-b2f7-53102bef36f3 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:20.005584 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1216/2447] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:53:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:53:20.543113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8e2be-9cfa-44d0-914a-f1d7153bbfb9 service placement] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:20.558000 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8e2be-9cfa-44d0-914a-f1d7153bbfb9 service placement] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:20.565923 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce992c9-1397-4647-a6d0-edf8cf32efd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:20.566640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce992c9-1397-4647-a6d0-edf8cf32efd8 service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:20.566936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce992c9-1397-4647-a6d0-edf8cf32efd8 service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:20.567385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce992c9-1397-4647-a6d0-edf8cf32efd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:20.569428 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-5ce992c9-1397-4647-a6d0-edf8cf32efd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:20.584275 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce992c9-1397-4647-a6d0-edf8cf32efd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:20.586125 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce992c9-1397-4647-a6d0-edf8cf32efd8 service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:20.601124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1232/2448] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:53:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:53:25.632106 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e4236e-391a-4000-9e74-72a0c9de8321 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:25.632566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e4236e-391a-4000-9e74-72a0c9de8321 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:25.632677 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e4236e-391a-4000-9e74-72a0c9de8321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:25.660770 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-a6e4236e-391a-4000-9e74-72a0c9de8321 None None] MFA Rules not processed for user `034858240ac34e549aa5e0754726f9c6`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:53:25.663920 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a6e4236e-391a-4000-9e74-72a0c9de8321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:25.677645 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1217/2449] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:53:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:53:26.611291 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac20de-5827-4fd1-adcf-795ddd8afd45 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:26.612260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac20de-5827-4fd1-adcf-795ddd8afd45 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:26.612550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac20de-5827-4fd1-adcf-795ddd8afd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:26.630813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-74ac20de-5827-4fd1-adcf-795ddd8afd45 None None] MFA Rules not processed for user `4d7c1adbb2ad456eaf5793a15d6fcb4b`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:53:26.632085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-74ac20de-5827-4fd1-adcf-795ddd8afd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:26.639145 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1233/2450] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:53:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:53:26.988454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e4236e-391a-4000-9e74-72a0c9de8321 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:26.998843 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e4236e-391a-4000-9e74-72a0c9de8321 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:27.004449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f128d0a-0126-44d6-98c7-66c648636aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:27.005565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f128d0a-0126-44d6-98c7-66c648636aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:27.005967 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f128d0a-0126-44d6-98c7-66c648636aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:27.006248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f128d0a-0126-44d6-98c7-66c648636aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:27.008109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-9f128d0a-0126-44d6-98c7-66c648636aa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:27.020431 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f128d0a-0126-44d6-98c7-66c648636aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:27.022229 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f128d0a-0126-44d6-98c7-66c648636aa0 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:27.031012 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1218/2451] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:53:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:53:31.910237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac20de-5827-4fd1-adcf-795ddd8afd45 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:31.924692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac20de-5827-4fd1-adcf-795ddd8afd45 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:31.936708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f454f-55a3-4618-acf3-0caa4ec045e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:31.937490 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f454f-55a3-4618-acf3-0caa4ec045e0 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:31.937654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f454f-55a3-4618-acf3-0caa4ec045e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:31.938117 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f454f-55a3-4618-acf3-0caa4ec045e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:31.944841 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-0c0f454f-55a3-4618-acf3-0caa4ec045e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:32.043124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f454f-55a3-4618-acf3-0caa4ec045e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:32.045490 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f454f-55a3-4618-acf3-0caa4ec045e0 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:32.063804 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1234/2452] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:53:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:53:32.123019 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e611d8-8fed-4cb9-a120-123c12ee7abf None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:32.123976 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e611d8-8fed-4cb9-a120-123c12ee7abf None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:32.124468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e611d8-8fed-4cb9-a120-123c12ee7abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:32.131253 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=66 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:32.146822 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-05e611d8-8fed-4cb9-a120-123c12ee7abf None None] MFA Rules not processed for user `955c44ffe1bd426e900c7405df44d1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:53:32.148282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-05e611d8-8fed-4cb9-a120-123c12ee7abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:32.155970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1219/2453] 10.222.0.36 () {58 vars in 967 bytes} [Wed Jun 22 11:53:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:53:32.217638 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f454f-55a3-4618-acf3-0caa4ec045e0 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:32.231469 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f454f-55a3-4618-acf3-0caa4ec045e0 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:32.238848 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6027a-608a-40a2-990b-3eb075dcc13f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd642e290e44e67b56eb3b8361d7af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:32.239878 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6027a-608a-40a2-990b-3eb075dcc13f service swift] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:32.240806 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6027a-608a-40a2-990b-3eb075dcc13f service swift] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:32.242187 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6027a-608a-40a2-990b-3eb075dcc13f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:32.253777 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f6027a-608a-40a2-990b-3eb075dcc13f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:32.255628 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f6027a-608a-40a2-990b-3eb075dcc13f service swift] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:32.266818 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1235/2454] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jun 22 11:53:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 22 11:53:32.710597 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e611d8-8fed-4cb9-a120-123c12ee7abf None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:32.721144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e611d8-8fed-4cb9-a120-123c12ee7abf None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:32.725270 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9bf87c-14e0-45e0-b714-3cbd93a5c23d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:32.726262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bf87c-14e0-45e0-b714-3cbd93a5c23d service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:32.726646 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bf87c-14e0-45e0-b714-3cbd93a5c23d service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:32.727185 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bf87c-14e0-45e0-b714-3cbd93a5c23d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:32.729243 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-aa9bf87c-14e0-45e0-b714-3cbd93a5c23d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:32.743186 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9bf87c-14e0-45e0-b714-3cbd93a5c23d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:32.745050 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9bf87c-14e0-45e0-b714-3cbd93a5c23d service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:32.757155 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1220/2455] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:53:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:53:38.058244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6027a-608a-40a2-990b-3eb075dcc13f service swift] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:38.075040 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6027a-608a-40a2-990b-3eb075dcc13f service swift] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:38.097875 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7610d20-a40c-431d-bcec-ae1134f585a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93630ed5e6dc4a5f958bc7cf1a68a26c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:38.098755 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7610d20-a40c-431d-bcec-ae1134f585a9 service placement] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:38.099093 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7610d20-a40c-431d-bcec-ae1134f585a9 service placement] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:38.099301 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7610d20-a40c-431d-bcec-ae1134f585a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:38.104402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b7610d20-a40c-431d-bcec-ae1134f585a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:38.144310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7610d20-a40c-431d-bcec-ae1134f585a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:38.145989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7610d20-a40c-431d-bcec-ae1134f585a9 service placement] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:38.172454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1236/2456] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:53:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:53:38.844647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b24f0-3a7b-4628-a7eb-ace615f28d7a None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:38.845262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b24f0-3a7b-4628-a7eb-ace615f28d7a None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:38.846180 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b24f0-3a7b-4628-a7eb-ace615f28d7a None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:38.847715 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1221/2457] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jun 22 11:53:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 22 11:53:38.860475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95832e30-1761-4a86-81d0-6f2e44bdd805 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:38.860766 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95832e30-1761-4a86-81d0-6f2e44bdd805 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:38.861049 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95832e30-1761-4a86-81d0-6f2e44bdd805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:38.901466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-95832e30-1761-4a86-81d0-6f2e44bdd805 None None] MFA Rules not processed for user `93630ed5e6dc4a5f958bc7cf1a68a26c`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:53:38.902727 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-95832e30-1761-4a86-81d0-6f2e44bdd805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:38.913633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1237/2458] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jun 22 11:53:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:53:38.926557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b24f0-3a7b-4628-a7eb-ace615f28d7a None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:38.946562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b24f0-3a7b-4628-a7eb-ace615f28d7a None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:38.951871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f013a0c-f893-449d-b51e-0176b8c772b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93630ed5e6dc4a5f958bc7cf1a68a26c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:38.952529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f013a0c-f893-449d-b51e-0176b8c772b5 service placement] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:38.952810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f013a0c-f893-449d-b51e-0176b8c772b5 service placement] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:38.953260 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f013a0c-f893-449d-b51e-0176b8c772b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:38.963488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f013a0c-f893-449d-b51e-0176b8c772b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:38.965746 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f013a0c-f893-449d-b51e-0176b8c772b5 service placement] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:38.979551 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1222/2459] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:53:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:53:38.985647 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95832e30-1761-4a86-81d0-6f2e44bdd805 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:38.987697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-95832e30-1761-4a86-81d0-6f2e44bdd805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:39.006782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95832e30-1761-4a86-81d0-6f2e44bdd805 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:39.013179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4cb0a-2887-44fc-9dab-553539df5fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:39.014985 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4cb0a-2887-44fc-9dab-553539df5fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:39.015514 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4cb0a-2887-44fc-9dab-553539df5fad admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:39.016202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4cb0a-2887-44fc-9dab-553539df5fad admin admin] PATH_INFO: `/v3/users/27cc3d38926f470c91eeb6aecfc86a48` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:39.018984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4cb0a-2887-44fc-9dab-553539df5fad admin admin] RBAC: Authorizing `identity:delete_user(user_id=27cc3d38926f470c91eeb6aecfc86a48)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:39.021768 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4cb0a-2887-44fc-9dab-553539df5fad admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:39.092716 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c6f4cb0a-2887-44fc-9dab-553539df5fad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27cc3d38926f470c91eeb6aecfc86a48'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:39.117255 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c6f4cb0a-2887-44fc-9dab-553539df5fad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27cc3d38926f470c91eeb6aecfc86a48'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:39.122485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:39.136151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c6f4cb0a-2887-44fc-9dab-553539df5fad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27cc3d38926f470c91eeb6aecfc86a48'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:39.145236 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c6f4cb0a-2887-44fc-9dab-553539df5fad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27cc3d38926f470c91eeb6aecfc86a48'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:39.146033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c6f4cb0a-2887-44fc-9dab-553539df5fad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27cc3d38926f470c91eeb6aecfc86a48'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:39.155438 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1238/2460] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:53:38 2022] DELETE /identity/v3/users/27cc3d38926f470c91eeb6aecfc86a48 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:53:39.502609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f013a0c-f893-449d-b51e-0176b8c772b5 service placement] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:39.515767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f013a0c-f893-449d-b51e-0176b8c772b5 service placement] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:39.592917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9abb3-373c-4952-aad1-375b612f36aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:39.593800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9abb3-373c-4952-aad1-375b612f36aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:39.594088 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9abb3-373c-4952-aad1-375b612f36aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:39.594372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9abb3-373c-4952-aad1-375b612f36aa admin admin] PATH_INFO: `/v3/projects/d20536d2fa444e97b7184c10e52d5cc5` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:39.596170 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9abb3-373c-4952-aad1-375b612f36aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d20536d2fa444e97b7184c10e52d5cc5)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:39.598083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f9abb3-373c-4952-aad1-375b612f36aa admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:39.653660 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-91f9abb3-373c-4952-aad1-375b612f36aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd20536d2fa444e97b7184c10e52d5cc5', 'request_id': 'req-91f9abb3-373c-4952-aad1-375b612f36aa'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:39.664897 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-91f9abb3-373c-4952-aad1-375b612f36aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd20536d2fa444e97b7184c10e52d5cc5', 'request_id': 'req-91f9abb3-373c-4952-aad1-375b612f36aa'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:39.674060 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-91f9abb3-373c-4952-aad1-375b612f36aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd20536d2fa444e97b7184c10e52d5cc5', 'request_id': 'req-91f9abb3-373c-4952-aad1-375b612f36aa'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:39.689678 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-91f9abb3-373c-4952-aad1-375b612f36aa admin admin] The token cache is being invalidate because project d20536d2fa444e97b7184c10e52d5cc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:53:39.690357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-91f9abb3-373c-4952-aad1-375b612f36aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:39.692146 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-91f9abb3-373c-4952-aad1-375b612f36aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:39.694987 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1223/2461] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:53:39 2022] DELETE /identity/v3/projects/d20536d2fa444e97b7184c10e52d5cc5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:53:39.700099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1d918-2ab7-4fd0-ba0a-70f0a2d84a42 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:39.700393 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1d918-2ab7-4fd0-ba0a-70f0a2d84a42 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:39.700673 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1d918-2ab7-4fd0-ba0a-70f0a2d84a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:39.726477 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f4e1d918-2ab7-4fd0-ba0a-70f0a2d84a42 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:53:39.727871 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f4e1d918-2ab7-4fd0-ba0a-70f0a2d84a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:39.830446 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1239/2462] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:53:39.831159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442ed19-1637-4fd4-ac6a-f4c9da927473 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:39.831617 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442ed19-1637-4fd4-ac6a-f4c9da927473 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:39.833650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442ed19-1637-4fd4-ac6a-f4c9da927473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:39.869460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-4442ed19-1637-4fd4-ac6a-f4c9da927473 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:53:39.870805 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-4442ed19-1637-4fd4-ac6a-f4c9da927473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:39.880750 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1224/2463] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:53:39.884700 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1d918-2ab7-4fd0-ba0a-70f0a2d84a42 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:39.897362 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1d918-2ab7-4fd0-ba0a-70f0a2d84a42 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:39.904697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990f2c4-92ab-49c2-9064-50c7d00d3c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:39.906162 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990f2c4-92ab-49c2-9064-50c7d00d3c64 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:39.906644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990f2c4-92ab-49c2-9064-50c7d00d3c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:39.908376 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990f2c4-92ab-49c2-9064-50c7d00d3c64 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:39.909441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990f2c4-92ab-49c2-9064-50c7d00d3c64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:53:39.909910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990f2c4-92ab-49c2-9064-50c7d00d3c64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:39.913585 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990f2c4-92ab-49c2-9064-50c7d00d3c64 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:39.934505 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1240/2464] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:53:39.939823 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442ed19-1637-4fd4-ac6a-f4c9da927473 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:39.952012 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442ed19-1637-4fd4-ac6a-f4c9da927473 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:39.957373 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b5912-1f79-4c1f-aa2d-b1fa0af2af6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:39.958043 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b5912-1f79-4c1f-aa2d-b1fa0af2af6d admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:39.958328 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b5912-1f79-4c1f-aa2d-b1fa0af2af6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:39.958607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b5912-1f79-4c1f-aa2d-b1fa0af2af6d admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:39.959660 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b5912-1f79-4c1f-aa2d-b1fa0af2af6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:39.962475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b5912-1f79-4c1f-aa2d-b1fa0af2af6d admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:40.008550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1225/2465] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:53:39 2022] POST /identity/v3/projects => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:53:40.012633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990f2c4-92ab-49c2-9064-50c7d00d3c64 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:40.025118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990f2c4-92ab-49c2-9064-50c7d00d3c64 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:40.091285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23920896-a80e-4c93-9166-29103fa0d39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:40.092179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23920896-a80e-4c93-9166-29103fa0d39d admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:40.092352 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23920896-a80e-4c93-9166-29103fa0d39d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:40.092762 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23920896-a80e-4c93-9166-29103fa0d39d admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:40.093469 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23920896-a80e-4c93-9166-29103fa0d39d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:40.097559 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23920896-a80e-4c93-9166-29103fa0d39d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:40.121072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1241/2466] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:53:40 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:53:40.125494 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b5912-1f79-4c1f-aa2d-b1fa0af2af6d admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:40.137608 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b5912-1f79-4c1f-aa2d-b1fa0af2af6d admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:40.143068 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90bcac-a7c1-41f3-be23-2ff2b2c14611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:40.143877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90bcac-a7c1-41f3-be23-2ff2b2c14611 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:40.144226 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90bcac-a7c1-41f3-be23-2ff2b2c14611 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:40.144762 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90bcac-a7c1-41f3-be23-2ff2b2c14611 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:40.145569 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90bcac-a7c1-41f3-be23-2ff2b2c14611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:40.147249 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90bcac-a7c1-41f3-be23-2ff2b2c14611 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:40.158765 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1226/2467] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:53:40.163016 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23920896-a80e-4c93-9166-29103fa0d39d admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:40.174623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23920896-a80e-4c93-9166-29103fa0d39d admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:40.182796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbd040-8993-4f95-95e0-214f9a932673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:40.184106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbd040-8993-4f95-95e0-214f9a932673 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:40.184232 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbd040-8993-4f95-95e0-214f9a932673 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:40.184457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbd040-8993-4f95-95e0-214f9a932673 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:40.185201 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fbd040-8993-4f95-95e0-214f9a932673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:40.186893 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fbd040-8993-4f95-95e0-214f9a932673 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:40.198993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1242/2468] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:53:40.203394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90bcac-a7c1-41f3-be23-2ff2b2c14611 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:40.213419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90bcac-a7c1-41f3-be23-2ff2b2c14611 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:40.224457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc8bde6-cdc6-4354-8a76-775ce4a8fb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:40.225790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8bde6-cdc6-4354-8a76-775ce4a8fb2e admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:40.226079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8bde6-cdc6-4354-8a76-775ce4a8fb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:40.226355 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8bde6-cdc6-4354-8a76-775ce4a8fb2e admin admin] PATH_INFO: `/v3/projects/343c05918eb94d418e4ece9ea17bde0b/users/8ef4031f7be54e07be13b65b5088ff13/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:40.235637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:40.244854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc8bde6-cdc6-4354-8a76-775ce4a8fb2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=343c05918eb94d418e4ece9ea17bde0b, user_id=8ef4031f7be54e07be13b65b5088ff13, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:40.249652 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc8bde6-cdc6-4354-8a76-775ce4a8fb2e admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:40.267396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1227/2469] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:53:40 2022] PUT /identity/v3/projects/343c05918eb94d418e4ece9ea17bde0b/users/8ef4031f7be54e07be13b65b5088ff13/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:53:40.277471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464c937-e43e-4320-abe4-236003b19cc6 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:40.277759 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464c937-e43e-4320-abe4-236003b19cc6 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:40.278208 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464c937-e43e-4320-abe4-236003b19cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:40.311454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-b464c937-e43e-4320-abe4-236003b19cc6 None None] MFA Rules not processed for user `8ef4031f7be54e07be13b65b5088ff13`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:53:40.312700 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b464c937-e43e-4320-abe4-236003b19cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:40.400616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:40.443051 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1243/2470] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:53:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:53:40.447341 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc8bde6-cdc6-4354-8a76-775ce4a8fb2e admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:40.626334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc8bde6-cdc6-4354-8a76-775ce4a8fb2e admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:40.633040 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e16d6d-8470-4b13-90ee-2562c486c3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:40.634140 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e16d6d-8470-4b13-90ee-2562c486c3dd admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:40.634700 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e16d6d-8470-4b13-90ee-2562c486c3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:40.635115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e16d6d-8470-4b13-90ee-2562c486c3dd admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:40.637019 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e16d6d-8470-4b13-90ee-2562c486c3dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:40.640027 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e16d6d-8470-4b13-90ee-2562c486c3dd admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:40.674443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1228/2471] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:53:40 2022] POST /identity/v3/projects => generated 381 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:53:40.678131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464c937-e43e-4320-abe4-236003b19cc6 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:40.699700 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464c937-e43e-4320-abe4-236003b19cc6 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:40.772920 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08fd38-8f0d-4400-ad38-a34ec285b562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:40.773383 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08fd38-8f0d-4400-ad38-a34ec285b562 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:40.774033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08fd38-8f0d-4400-ad38-a34ec285b562 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:40.774033 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08fd38-8f0d-4400-ad38-a34ec285b562 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:40.774829 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08fd38-8f0d-4400-ad38-a34ec285b562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:40.776906 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08fd38-8f0d-4400-ad38-a34ec285b562 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:40.806547 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e16d6d-8470-4b13-90ee-2562c486c3dd admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:40.807797 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1244/2472] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:53:40 2022] POST /identity/v3/users => generated 297 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:53:40.819930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e16d6d-8470-4b13-90ee-2562c486c3dd admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:40.826179 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0715f18-bed6-4d46-ac0f-c25a1d22b2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:40.827631 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0715f18-bed6-4d46-ac0f-c25a1d22b2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:40.828009 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0715f18-bed6-4d46-ac0f-c25a1d22b2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:40.828370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0715f18-bed6-4d46-ac0f-c25a1d22b2d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:40.829192 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0715f18-bed6-4d46-ac0f-c25a1d22b2d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:40.831510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0715f18-bed6-4d46-ac0f-c25a1d22b2d5 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:40.851939 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08fd38-8f0d-4400-ad38-a34ec285b562 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:40.852317 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1229/2473] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:53:40.863623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08fd38-8f0d-4400-ad38-a34ec285b562 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:40.868585 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906c256-13b7-4081-8e90-a09ee7bfa221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:40.869540 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906c256-13b7-4081-8e90-a09ee7bfa221 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:40.869834 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906c256-13b7-4081-8e90-a09ee7bfa221 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:40.870116 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906c256-13b7-4081-8e90-a09ee7bfa221 admin admin] PATH_INFO: `/v3/projects/193041babc604a37ac3d602710590d67/users/0fa9d98424784d36b851d41d68a2b5ac/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:40.894591 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906c256-13b7-4081-8e90-a09ee7bfa221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=193041babc604a37ac3d602710590d67, user_id=0fa9d98424784d36b851d41d68a2b5ac, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:40.898413 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906c256-13b7-4081-8e90-a09ee7bfa221 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:40.920200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1245/2474] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:53:40 2022] PUT /identity/v3/projects/193041babc604a37ac3d602710590d67/users/0fa9d98424784d36b851d41d68a2b5ac/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:53:40.921973 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0715f18-bed6-4d46-ac0f-c25a1d22b2d5 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:40.944538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0715f18-bed6-4d46-ac0f-c25a1d22b2d5 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:41.024279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9cb7f-5608-4cbc-b588-ab1919f9e78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:41.025278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9cb7f-5608-4cbc-b588-ab1919f9e78c admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:41.025606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9cb7f-5608-4cbc-b588-ab1919f9e78c admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:41.026012 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9cb7f-5608-4cbc-b588-ab1919f9e78c admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:41.026861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9cb7f-5608-4cbc-b588-ab1919f9e78c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:41.029585 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9cb7f-5608-4cbc-b588-ab1919f9e78c admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:41.053856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1230/2475] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:53:41.054612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906c256-13b7-4081-8e90-a09ee7bfa221 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:41.071637 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906c256-13b7-4081-8e90-a09ee7bfa221 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:41.087441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2e547-3fc3-497e-8a5b-58cc97c847fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:41.088672 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2e547-3fc3-497e-8a5b-58cc97c847fe admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:41.092494 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2e547-3fc3-497e-8a5b-58cc97c847fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:41.092800 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2e547-3fc3-497e-8a5b-58cc97c847fe admin admin] PATH_INFO: `/v3/projects/193041babc604a37ac3d602710590d67/users/0fa9d98424784d36b851d41d68a2b5ac/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:41.096806 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f2e547-3fc3-497e-8a5b-58cc97c847fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=193041babc604a37ac3d602710590d67, user_id=0fa9d98424784d36b851d41d68a2b5ac, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:41.100022 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f2e547-3fc3-497e-8a5b-58cc97c847fe admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:41.113116 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1246/2476] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:53:41 2022] PUT /identity/v3/projects/193041babc604a37ac3d602710590d67/users/0fa9d98424784d36b851d41d68a2b5ac/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:53:41.124198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbebc1-6bfc-461b-9dbb-c094c7c7ca90 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:41.124497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbebc1-6bfc-461b-9dbb-c094c7c7ca90 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:41.124787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbebc1-6bfc-461b-9dbb-c094c7c7ca90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:41.158707 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-89dbebc1-6bfc-461b-9dbb-c094c7c7ca90 None None] MFA Rules not processed for user `0fa9d98424784d36b851d41d68a2b5ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:53:41.160122 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-89dbebc1-6bfc-461b-9dbb-c094c7c7ca90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:41.220974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:41.252539 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1231/2477] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:53:41 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:53:41.264516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2e547-3fc3-497e-8a5b-58cc97c847fe admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:41.266376 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-75f2e547-3fc3-497e-8a5b-58cc97c847fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:41.361002 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2e547-3fc3-497e-8a5b-58cc97c847fe admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:41.366319 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ffe36-f83f-4d88-8bc3-b1496f51c63d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:41.367501 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ffe36-f83f-4d88-8bc3-b1496f51c63d service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:41.367805 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ffe36-f83f-4d88-8bc3-b1496f51c63d service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:41.368081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ffe36-f83f-4d88-8bc3-b1496f51c63d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:41.380677 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ffe36-f83f-4d88-8bc3-b1496f51c63d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:41.382315 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ffe36-f83f-4d88-8bc3-b1496f51c63d service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:41.394237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1247/2478] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:53:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:53:41.489849 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dbebc1-6bfc-461b-9dbb-c094c7c7ca90 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:41.500676 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dbebc1-6bfc-461b-9dbb-c094c7c7ca90 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:41.505422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb566565-5061-4266-a8a8-92fa12daf843 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:41.506092 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb566565-5061-4266-a8a8-92fa12daf843 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:41.506380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb566565-5061-4266-a8a8-92fa12daf843 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:41.506775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb566565-5061-4266-a8a8-92fa12daf843 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:41.576255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb566565-5061-4266-a8a8-92fa12daf843 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:41.577618 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb566565-5061-4266-a8a8-92fa12daf843 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:41.587401 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1232/2479] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:53:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:53:42.312316 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ffe36-f83f-4d88-8bc3-b1496f51c63d service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:42.314304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-8c6ffe36-f83f-4d88-8bc3-b1496f51c63d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:42.335051 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:42.385296 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ffe36-f83f-4d88-8bc3-b1496f51c63d service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:42.391240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2c52f-71b1-4b1c-bc5d-d556be198096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:42.392782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2c52f-71b1-4b1c-bc5d-d556be198096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:42.393103 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2c52f-71b1-4b1c-bc5d-d556be198096 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:42.393386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2c52f-71b1-4b1c-bc5d-d556be198096 admin admin] PATH_INFO: `/v3/users/8fc6cd29af4d4c28bec9bf6cc9db5749` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:42.394651 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2c52f-71b1-4b1c-bc5d-d556be198096 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fc6cd29af4d4c28bec9bf6cc9db5749)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:42.396706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2c52f-71b1-4b1c-bc5d-d556be198096 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:42.448392 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-dbb2c52f-71b1-4b1c-bc5d-d556be198096 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fc6cd29af4d4c28bec9bf6cc9db5749'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:42.461382 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-dbb2c52f-71b1-4b1c-bc5d-d556be198096 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fc6cd29af4d4c28bec9bf6cc9db5749'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:42.475509 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-dbb2c52f-71b1-4b1c-bc5d-d556be198096 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fc6cd29af4d4c28bec9bf6cc9db5749'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:42.483714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-dbb2c52f-71b1-4b1c-bc5d-d556be198096 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fc6cd29af4d4c28bec9bf6cc9db5749'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:42.484398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-dbb2c52f-71b1-4b1c-bc5d-d556be198096 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fc6cd29af4d4c28bec9bf6cc9db5749'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:42.491393 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1248/2480] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:53:42 2022] DELETE /identity/v3/users/8fc6cd29af4d4c28bec9bf6cc9db5749 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:53:42.671212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb566565-5061-4266-a8a8-92fa12daf843 service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:42.683434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb566565-5061-4266-a8a8-92fa12daf843 service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:42.740415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479bcb2-953f-4aa3-8307-81b44b1b1c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:42.741292 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479bcb2-953f-4aa3-8307-81b44b1b1c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:42.741557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479bcb2-953f-4aa3-8307-81b44b1b1c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:42.741818 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479bcb2-953f-4aa3-8307-81b44b1b1c76 admin admin] PATH_INFO: `/v3/projects/2975e2c648284a80824126eecc5d0269` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:42.743315 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479bcb2-953f-4aa3-8307-81b44b1b1c76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2975e2c648284a80824126eecc5d0269)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:42.745355 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479bcb2-953f-4aa3-8307-81b44b1b1c76 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:42.795912 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b479bcb2-953f-4aa3-8307-81b44b1b1c76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2975e2c648284a80824126eecc5d0269', 'request_id': 'req-b479bcb2-953f-4aa3-8307-81b44b1b1c76'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:42.803658 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b479bcb2-953f-4aa3-8307-81b44b1b1c76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2975e2c648284a80824126eecc5d0269', 'request_id': 'req-b479bcb2-953f-4aa3-8307-81b44b1b1c76'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:42.810365 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b479bcb2-953f-4aa3-8307-81b44b1b1c76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2975e2c648284a80824126eecc5d0269', 'request_id': 'req-b479bcb2-953f-4aa3-8307-81b44b1b1c76'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:42.823860 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b479bcb2-953f-4aa3-8307-81b44b1b1c76 admin admin] The token cache is being invalidate because project 2975e2c648284a80824126eecc5d0269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:53:42.824250 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b479bcb2-953f-4aa3-8307-81b44b1b1c76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:42.825207 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b479bcb2-953f-4aa3-8307-81b44b1b1c76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:42.827753 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1233/2481] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:53:42 2022] DELETE /identity/v3/projects/2975e2c648284a80824126eecc5d0269 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:53:42.832194 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2c52f-71b1-4b1c-bc5d-d556be198096 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:42.833677 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-dbb2c52f-71b1-4b1c-bc5d-d556be198096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:42.848385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2c52f-71b1-4b1c-bc5d-d556be198096 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:42.924515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114dfe50-878d-42f6-ad10-56bf704a7cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:42.925399 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dfe50-878d-42f6-ad10-56bf704a7cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:42.925748 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dfe50-878d-42f6-ad10-56bf704a7cac admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:42.926122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dfe50-878d-42f6-ad10-56bf704a7cac admin admin] PATH_INFO: `/v3/users/8819384277834fe795cc0017d1ff29c4` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:42.931281 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114dfe50-878d-42f6-ad10-56bf704a7cac admin admin] RBAC: Authorizing `identity:delete_user(user_id=8819384277834fe795cc0017d1ff29c4)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:42.931281 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114dfe50-878d-42f6-ad10-56bf704a7cac admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:42.986253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:42.995169 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-114dfe50-878d-42f6-ad10-56bf704a7cac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8819384277834fe795cc0017d1ff29c4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:43.014190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-114dfe50-878d-42f6-ad10-56bf704a7cac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8819384277834fe795cc0017d1ff29c4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:43.028715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-114dfe50-878d-42f6-ad10-56bf704a7cac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8819384277834fe795cc0017d1ff29c4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:43.041901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-114dfe50-878d-42f6-ad10-56bf704a7cac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8819384277834fe795cc0017d1ff29c4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:43.042934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-114dfe50-878d-42f6-ad10-56bf704a7cac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8819384277834fe795cc0017d1ff29c4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:43.055060 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1249/2482] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:53:42 2022] DELETE /identity/v3/users/8819384277834fe795cc0017d1ff29c4 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:53:43.300880 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479bcb2-953f-4aa3-8307-81b44b1b1c76 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:43.302359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b479bcb2-953f-4aa3-8307-81b44b1b1c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:43.321993 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479bcb2-953f-4aa3-8307-81b44b1b1c76 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:43.330773 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:43.382696 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71ec4b-8cff-4b96-af42-43b3fa188d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:43.383527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71ec4b-8cff-4b96-af42-43b3fa188d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:43.383813 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71ec4b-8cff-4b96-af42-43b3fa188d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:43.384095 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71ec4b-8cff-4b96-af42-43b3fa188d59 admin admin] PATH_INFO: `/v3/projects/e7f491d21b9f4d2795facfff24b32526` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:43.385299 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c71ec4b-8cff-4b96-af42-43b3fa188d59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7f491d21b9f4d2795facfff24b32526)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:43.387225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c71ec4b-8cff-4b96-af42-43b3fa188d59 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:43.433263 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4c71ec4b-8cff-4b96-af42-43b3fa188d59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7f491d21b9f4d2795facfff24b32526', 'request_id': 'req-4c71ec4b-8cff-4b96-af42-43b3fa188d59'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:43.441269 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4c71ec4b-8cff-4b96-af42-43b3fa188d59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7f491d21b9f4d2795facfff24b32526', 'request_id': 'req-4c71ec4b-8cff-4b96-af42-43b3fa188d59'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:43.447682 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4c71ec4b-8cff-4b96-af42-43b3fa188d59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7f491d21b9f4d2795facfff24b32526', 'request_id': 'req-4c71ec4b-8cff-4b96-af42-43b3fa188d59'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:43.457934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4c71ec4b-8cff-4b96-af42-43b3fa188d59 admin admin] The token cache is being invalidate because project e7f491d21b9f4d2795facfff24b32526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:53:43.458222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4c71ec4b-8cff-4b96-af42-43b3fa188d59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:43.458893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-4c71ec4b-8cff-4b96-af42-43b3fa188d59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:43.460477 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1234/2483] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:53:43 2022] DELETE /identity/v3/projects/e7f491d21b9f4d2795facfff24b32526 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:53:43.465512 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1afb73-e6f7-44eb-a262-f2387b446ee9 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:43.465961 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1afb73-e6f7-44eb-a262-f2387b446ee9 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:43.466081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1afb73-e6f7-44eb-a262-f2387b446ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:43.484546 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-3e1afb73-e6f7-44eb-a262-f2387b446ee9 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:53:43.487432 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-3e1afb73-e6f7-44eb-a262-f2387b446ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:43.549003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1250/2484] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:53:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:53:43.557018 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eec627-1ae7-4572-b4ab-294764ec711e None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:43.557310 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eec627-1ae7-4572-b4ab-294764ec711e None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:43.557590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eec627-1ae7-4572-b4ab-294764ec711e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:43.575917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-92eec627-1ae7-4572-b4ab-294764ec711e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:53:43.577118 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-92eec627-1ae7-4572-b4ab-294764ec711e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:43.585421 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1235/2485] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:53:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:53:43.589475 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1afb73-e6f7-44eb-a262-f2387b446ee9 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:43.599965 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1afb73-e6f7-44eb-a262-f2387b446ee9 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:43.604732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb99f72-60a0-47fa-80aa-9eec3fe5c088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:43.605392 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb99f72-60a0-47fa-80aa-9eec3fe5c088 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:43.605678 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb99f72-60a0-47fa-80aa-9eec3fe5c088 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:43.606187 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb99f72-60a0-47fa-80aa-9eec3fe5c088 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:43.606949 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb99f72-60a0-47fa-80aa-9eec3fe5c088 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:53:43.607258 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb99f72-60a0-47fa-80aa-9eec3fe5c088 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:43.608978 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb99f72-60a0-47fa-80aa-9eec3fe5c088 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:43.622002 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1251/2486] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:53:43.626486 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eec627-1ae7-4572-b4ab-294764ec711e None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:43.636109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eec627-1ae7-4572-b4ab-294764ec711e None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:43.641226 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df85e87-f738-42b8-bc39-98b2ac43fa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:43.641990 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df85e87-f738-42b8-bc39-98b2ac43fa7a admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:43.642358 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df85e87-f738-42b8-bc39-98b2ac43fa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:43.642720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df85e87-f738-42b8-bc39-98b2ac43fa7a admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:43.643521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df85e87-f738-42b8-bc39-98b2ac43fa7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:43.645517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df85e87-f738-42b8-bc39-98b2ac43fa7a admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:43.679028 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1236/2487] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:53:43 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:53:43.680406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb99f72-60a0-47fa-80aa-9eec3fe5c088 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:43.691176 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb99f72-60a0-47fa-80aa-9eec3fe5c088 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:43.753439 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccb03d-7be8-459a-bfd2-0bd1829bc413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:43.754146 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccb03d-7be8-459a-bfd2-0bd1829bc413 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:43.754439 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccb03d-7be8-459a-bfd2-0bd1829bc413 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:43.754703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccb03d-7be8-459a-bfd2-0bd1829bc413 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:43.755430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cccb03d-7be8-459a-bfd2-0bd1829bc413 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:43.757363 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cccb03d-7be8-459a-bfd2-0bd1829bc413 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:43.777591 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1252/2488] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:53:43 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:53:43.781527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df85e87-f738-42b8-bc39-98b2ac43fa7a admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:43.795392 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df85e87-f738-42b8-bc39-98b2ac43fa7a admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:43.801150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a91383-32a6-4446-8d61-3a7179075423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:43.801818 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a91383-32a6-4446-8d61-3a7179075423 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:43.802099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a91383-32a6-4446-8d61-3a7179075423 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:43.802371 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a91383-32a6-4446-8d61-3a7179075423 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:43.803094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a91383-32a6-4446-8d61-3a7179075423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:43.804707 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a91383-32a6-4446-8d61-3a7179075423 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:43.815852 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1237/2489] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:53:43.819872 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccb03d-7be8-459a-bfd2-0bd1829bc413 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:43.830830 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cccb03d-7be8-459a-bfd2-0bd1829bc413 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:43.835563 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93cb0ab-4831-43f7-a6fb-6e4726fdb0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:43.836242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cb0ab-4831-43f7-a6fb-6e4726fdb0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:43.836523 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cb0ab-4831-43f7-a6fb-6e4726fdb0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:43.836799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cb0ab-4831-43f7-a6fb-6e4726fdb0a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:43.837540 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93cb0ab-4831-43f7-a6fb-6e4726fdb0a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:43.839195 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93cb0ab-4831-43f7-a6fb-6e4726fdb0a3 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:43.850886 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1253/2490] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:53:43.855408 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a91383-32a6-4446-8d61-3a7179075423 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:43.865704 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a91383-32a6-4446-8d61-3a7179075423 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:43.870688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e778d956-afea-4e4a-8932-0d51ccd8efe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:43.871601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778d956-afea-4e4a-8932-0d51ccd8efe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:43.871883 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778d956-afea-4e4a-8932-0d51ccd8efe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:43.872162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778d956-afea-4e4a-8932-0d51ccd8efe4 admin admin] PATH_INFO: `/v3/projects/edbd324693e24bcdbf41c207d29d7e2e/users/239de565f79f474bb21699fd9adc96c4/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:43.888951 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e778d956-afea-4e4a-8932-0d51ccd8efe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edbd324693e24bcdbf41c207d29d7e2e, user_id=239de565f79f474bb21699fd9adc96c4, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:43.891568 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e778d956-afea-4e4a-8932-0d51ccd8efe4 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:43.906003 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1238/2491] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:53:43 2022] PUT /identity/v3/projects/edbd324693e24bcdbf41c207d29d7e2e/users/239de565f79f474bb21699fd9adc96c4/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:53:43.911787 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6626937-0e84-41e2-8355-fe38c61f236b None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:43.912075 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6626937-0e84-41e2-8355-fe38c61f236b None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:43.912352 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6626937-0e84-41e2-8355-fe38c61f236b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:43.930177 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-c6626937-0e84-41e2-8355-fe38c61f236b None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:53:43.931426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-c6626937-0e84-41e2-8355-fe38c61f236b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:43.956099 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:44.001426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1254/2492] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:53:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:53:44.011185 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e778d956-afea-4e4a-8932-0d51ccd8efe4 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:44.013297 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-e778d956-afea-4e4a-8932-0d51ccd8efe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:44.060936 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:44.082632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e778d956-afea-4e4a-8932-0d51ccd8efe4 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:44.086581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be623629-6376-4ec5-b8fc-d01b02c7faef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:44.087257 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be623629-6376-4ec5-b8fc-d01b02c7faef service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:44.087602 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be623629-6376-4ec5-b8fc-d01b02c7faef service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:44.087829 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be623629-6376-4ec5-b8fc-d01b02c7faef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:44.098637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be623629-6376-4ec5-b8fc-d01b02c7faef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:44.100127 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be623629-6376-4ec5-b8fc-d01b02c7faef service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:44.110630 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1239/2493] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:53:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:53:48.664102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5f23b-39dd-4498-ab6c-282d41558c61 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:48.665039 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5f23b-39dd-4498-ab6c-282d41558c61 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:48.665478 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5f23b-39dd-4498-ab6c-282d41558c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:48.710523 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-26f5f23b-39dd-4498-ab6c-282d41558c61 None None] MFA Rules not processed for user `239de565f79f474bb21699fd9adc96c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:53:48.711952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-26f5f23b-39dd-4498-ab6c-282d41558c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:48.813051 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1255/2494] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:53:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:53:48.813856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be623629-6376-4ec5-b8fc-d01b02c7faef service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:48.828897 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be623629-6376-4ec5-b8fc-d01b02c7faef service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:48.834646 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21e56a-d481-44c6-b81b-c7f9d304fbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:48.835317 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21e56a-d481-44c6-b81b-c7f9d304fbc4 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:48.835633 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21e56a-d481-44c6-b81b-c7f9d304fbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:48.835910 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21e56a-d481-44c6-b81b-c7f9d304fbc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:48.836600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21e56a-d481-44c6-b81b-c7f9d304fbc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:48.838524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21e56a-d481-44c6-b81b-c7f9d304fbc4 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:48.868361 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1240/2495] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:53:48 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:53:48.872436 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5f23b-39dd-4498-ab6c-282d41558c61 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:48.883951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5f23b-39dd-4498-ab6c-282d41558c61 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:48.962072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39f06d-a7ba-473c-ba1e-cb03c8d28d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:48.962757 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39f06d-a7ba-473c-ba1e-cb03c8d28d6d admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:48.963043 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39f06d-a7ba-473c-ba1e-cb03c8d28d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:48.963318 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39f06d-a7ba-473c-ba1e-cb03c8d28d6d admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:48.964070 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39f06d-a7ba-473c-ba1e-cb03c8d28d6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:48.966021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39f06d-a7ba-473c-ba1e-cb03c8d28d6d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:48.989680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1256/2496] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:53:48 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:53:48.993633 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21e56a-d481-44c6-b81b-c7f9d304fbc4 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:49.004803 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21e56a-d481-44c6-b81b-c7f9d304fbc4 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:49.010460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66594d0e-6d87-44d4-ac59-2e26e6f1df5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:49.011410 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66594d0e-6d87-44d4-ac59-2e26e6f1df5a admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:49.011696 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66594d0e-6d87-44d4-ac59-2e26e6f1df5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:49.011973 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66594d0e-6d87-44d4-ac59-2e26e6f1df5a admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:49.013661 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66594d0e-6d87-44d4-ac59-2e26e6f1df5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:49.015496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66594d0e-6d87-44d4-ac59-2e26e6f1df5a admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:49.027438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1241/2497] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:53:49.031288 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39f06d-a7ba-473c-ba1e-cb03c8d28d6d admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:49.042423 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39f06d-a7ba-473c-ba1e-cb03c8d28d6d admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:49.048586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8403f-cf36-4c67-97c6-14b8069057e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:49.049640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8403f-cf36-4c67-97c6-14b8069057e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:49.049927 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8403f-cf36-4c67-97c6-14b8069057e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:49.050206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8403f-cf36-4c67-97c6-14b8069057e2 admin admin] PATH_INFO: `/v3/projects/a71966a6b5ca4471892906c483adf73d/users/258c84e10fd3449794752d64f56d41ec/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:49.072894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8403f-cf36-4c67-97c6-14b8069057e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71966a6b5ca4471892906c483adf73d, user_id=258c84e10fd3449794752d64f56d41ec, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:49.076834 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8403f-cf36-4c67-97c6-14b8069057e2 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:49.093570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1257/2498] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:53:49 2022] PUT /identity/v3/projects/a71966a6b5ca4471892906c483adf73d/users/258c84e10fd3449794752d64f56d41ec/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:53:49.099896 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66594d0e-6d87-44d4-ac59-2e26e6f1df5a admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:49.111567 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66594d0e-6d87-44d4-ac59-2e26e6f1df5a admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:49.238407 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16cbae8-92f4-41da-8370-e8b6bd4c3b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:49.239270 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16cbae8-92f4-41da-8370-e8b6bd4c3b29 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:49.240413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16cbae8-92f4-41da-8370-e8b6bd4c3b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:49.240413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16cbae8-92f4-41da-8370-e8b6bd4c3b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:49.240904 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16cbae8-92f4-41da-8370-e8b6bd4c3b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:49.242635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16cbae8-92f4-41da-8370-e8b6bd4c3b29 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:49.258962 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1242/2499] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:53:49.260562 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8403f-cf36-4c67-97c6-14b8069057e2 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:49.272160 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8403f-cf36-4c67-97c6-14b8069057e2 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:49.310811 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e3cc3-df98-430b-b4ec-66b566818c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:49.311685 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e3cc3-df98-430b-b4ec-66b566818c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:49.311973 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e3cc3-df98-430b-b4ec-66b566818c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:49.312262 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e3cc3-df98-430b-b4ec-66b566818c55 admin admin] PATH_INFO: `/v3/projects/a71966a6b5ca4471892906c483adf73d/users/258c84e10fd3449794752d64f56d41ec/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:49.320293 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e3cc3-df98-430b-b4ec-66b566818c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71966a6b5ca4471892906c483adf73d, user_id=258c84e10fd3449794752d64f56d41ec, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:49.323635 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e3cc3-df98-430b-b4ec-66b566818c55 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:49.351763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1258/2500] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:53:49 2022] PUT /identity/v3/projects/a71966a6b5ca4471892906c483adf73d/users/258c84e10fd3449794752d64f56d41ec/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:53:53.243553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16cbae8-92f4-41da-8370-e8b6bd4c3b29 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:53.245688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c16cbae8-92f4-41da-8370-e8b6bd4c3b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:53.267473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16cbae8-92f4-41da-8370-e8b6bd4c3b29 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:53.358456 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8052e-ac8b-4070-a2de-f8c10c00a4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:53.359464 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8052e-ac8b-4070-a2de-f8c10c00a4b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:53.359812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8052e-ac8b-4070-a2de-f8c10c00a4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:53.360203 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8052e-ac8b-4070-a2de-f8c10c00a4b5 admin admin] PATH_INFO: `/v3/users/8ef4031f7be54e07be13b65b5088ff13` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:53.364067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8052e-ac8b-4070-a2de-f8c10c00a4b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ef4031f7be54e07be13b65b5088ff13)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:53.367949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8052e-ac8b-4070-a2de-f8c10c00a4b5 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:53.442229 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:53.446166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d1e8052e-ac8b-4070-a2de-f8c10c00a4b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ef4031f7be54e07be13b65b5088ff13'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:53.469606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d1e8052e-ac8b-4070-a2de-f8c10c00a4b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ef4031f7be54e07be13b65b5088ff13'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:53.483711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d1e8052e-ac8b-4070-a2de-f8c10c00a4b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ef4031f7be54e07be13b65b5088ff13'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:53.484406 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d1e8052e-ac8b-4070-a2de-f8c10c00a4b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ef4031f7be54e07be13b65b5088ff13'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:53.497433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d1e8052e-ac8b-4070-a2de-f8c10c00a4b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ef4031f7be54e07be13b65b5088ff13'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:53.508523 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1243/2501] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:53:53 2022] DELETE /identity/v3/users/8ef4031f7be54e07be13b65b5088ff13 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:53:53.510249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370833cc-53ff-42e0-9626-bf43b72b0609 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:53.510728 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370833cc-53ff-42e0-9626-bf43b72b0609 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:53.510833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370833cc-53ff-42e0-9626-bf43b72b0609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:53.526184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:53.542894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-370833cc-53ff-42e0-9626-bf43b72b0609 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:53:53.544179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-370833cc-53ff-42e0-9626-bf43b72b0609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:53.636966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1259/2502] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:53:53.646006 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8052e-ac8b-4070-a2de-f8c10c00a4b5 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:53.647563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d1e8052e-ac8b-4070-a2de-f8c10c00a4b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:53.731564 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8052e-ac8b-4070-a2de-f8c10c00a4b5 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:53.736469 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21667d7-f951-4ddd-8e7e-f205ea44ac7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:53.738586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21667d7-f951-4ddd-8e7e-f205ea44ac7b service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:53.739244 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21667d7-f951-4ddd-8e7e-f205ea44ac7b service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:53.739726 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21667d7-f951-4ddd-8e7e-f205ea44ac7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:53.757263 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21667d7-f951-4ddd-8e7e-f205ea44ac7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:53.758987 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21667d7-f951-4ddd-8e7e-f205ea44ac7b service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:53.784532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1244/2503] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:53:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:53:54.411768 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370833cc-53ff-42e0-9626-bf43b72b0609 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:54.413229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-370833cc-53ff-42e0-9626-bf43b72b0609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:54.438704 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370833cc-53ff-42e0-9626-bf43b72b0609 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:54.445524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c219d6c2-8cb1-491d-8941-084ecce382db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:54.446430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219d6c2-8cb1-491d-8941-084ecce382db admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:54.446712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219d6c2-8cb1-491d-8941-084ecce382db admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:54.446987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c219d6c2-8cb1-491d-8941-084ecce382db admin admin] PATH_INFO: `/v3/projects/343c05918eb94d418e4ece9ea17bde0b` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:54.449948 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219d6c2-8cb1-491d-8941-084ecce382db admin admin] RBAC: Authorizing `identity:delete_project(project_id=343c05918eb94d418e4ece9ea17bde0b)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:54.451931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c219d6c2-8cb1-491d-8941-084ecce382db admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:54.550818 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c219d6c2-8cb1-491d-8941-084ecce382db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '343c05918eb94d418e4ece9ea17bde0b', 'request_id': 'req-c219d6c2-8cb1-491d-8941-084ecce382db'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:54.561911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c219d6c2-8cb1-491d-8941-084ecce382db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '343c05918eb94d418e4ece9ea17bde0b', 'request_id': 'req-c219d6c2-8cb1-491d-8941-084ecce382db'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:54.575888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c219d6c2-8cb1-491d-8941-084ecce382db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '343c05918eb94d418e4ece9ea17bde0b', 'request_id': 'req-c219d6c2-8cb1-491d-8941-084ecce382db'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:54.591133 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c219d6c2-8cb1-491d-8941-084ecce382db admin admin] The token cache is being invalidate because project 343c05918eb94d418e4ece9ea17bde0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:53:54.591894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c219d6c2-8cb1-491d-8941-084ecce382db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:54.592816 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c219d6c2-8cb1-491d-8941-084ecce382db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:54.595390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1260/2504] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:53:54 2022] DELETE /identity/v3/projects/343c05918eb94d418e4ece9ea17bde0b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:53:54.599593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21667d7-f951-4ddd-8e7e-f205ea44ac7b service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:54.619102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21667d7-f951-4ddd-8e7e-f205ea44ac7b service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:54.704678 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60399105-3904-4c8d-bcf4-9298023aafb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:54.706474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60399105-3904-4c8d-bcf4-9298023aafb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:54.706985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60399105-3904-4c8d-bcf4-9298023aafb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:54.708688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60399105-3904-4c8d-bcf4-9298023aafb5 admin admin] PATH_INFO: `/v3/users/0fa9d98424784d36b851d41d68a2b5ac` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:54.712028 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60399105-3904-4c8d-bcf4-9298023aafb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fa9d98424784d36b851d41d68a2b5ac)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:54.713973 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60399105-3904-4c8d-bcf4-9298023aafb5 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:54.757402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf368bf-b8c5-42f9-aebd-8014e0fd0ef1 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:54.757778 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf368bf-b8c5-42f9-aebd-8014e0fd0ef1 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:54.758292 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf368bf-b8c5-42f9-aebd-8014e0fd0ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:54.791610 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-fdf368bf-b8c5-42f9-aebd-8014e0fd0ef1 None None] MFA Rules not processed for user `258c84e10fd3449794752d64f56d41ec`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:53:54.792891 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-60399105-3904-4c8d-bcf4-9298023aafb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fa9d98424784d36b851d41d68a2b5ac'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:54.793130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-fdf368bf-b8c5-42f9-aebd-8014e0fd0ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:54.808165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-60399105-3904-4c8d-bcf4-9298023aafb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fa9d98424784d36b851d41d68a2b5ac'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:54.811724 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:54.819531 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-60399105-3904-4c8d-bcf4-9298023aafb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fa9d98424784d36b851d41d68a2b5ac'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:54.820197 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-60399105-3904-4c8d-bcf4-9298023aafb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fa9d98424784d36b851d41d68a2b5ac'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:54.830581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-60399105-3904-4c8d-bcf4-9298023aafb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fa9d98424784d36b851d41d68a2b5ac'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:54.838220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1245/2505] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:53:54 2022] DELETE /identity/v3/users/0fa9d98424784d36b851d41d68a2b5ac => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:53:54.886328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1261/2506] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:53:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:53:54.894134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60399105-3904-4c8d-bcf4-9298023aafb5 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:54.896305 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-60399105-3904-4c8d-bcf4-9298023aafb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:55.019025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60399105-3904-4c8d-bcf4-9298023aafb5 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:55.025269 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83686b03-8a80-45f0-a81e-618fed5efc5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:55.025943 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83686b03-8a80-45f0-a81e-618fed5efc5e service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:55.026230 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83686b03-8a80-45f0-a81e-618fed5efc5e service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:55.026513 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83686b03-8a80-45f0-a81e-618fed5efc5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:55.028412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-83686b03-8a80-45f0-a81e-618fed5efc5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:55.107586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83686b03-8a80-45f0-a81e-618fed5efc5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:55.109198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83686b03-8a80-45f0-a81e-618fed5efc5e service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:55.121101 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1246/2507] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:53:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:53:55.192655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf368bf-b8c5-42f9-aebd-8014e0fd0ef1 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:55.194250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-fdf368bf-b8c5-42f9-aebd-8014e0fd0ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:55.220557 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf368bf-b8c5-42f9-aebd-8014e0fd0ef1 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:55.233252 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:55.301150 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e0afd-9ab1-40e2-b263-6c72a8f085a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:55.302028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e0afd-9ab1-40e2-b263-6c72a8f085a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:55.302308 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e0afd-9ab1-40e2-b263-6c72a8f085a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:55.302579 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e0afd-9ab1-40e2-b263-6c72a8f085a9 admin admin] PATH_INFO: `/v3/projects/193041babc604a37ac3d602710590d67` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:55.305521 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e0afd-9ab1-40e2-b263-6c72a8f085a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=193041babc604a37ac3d602710590d67)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:55.308327 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e0afd-9ab1-40e2-b263-6c72a8f085a9 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:55.373081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-cb6e0afd-9ab1-40e2-b263-6c72a8f085a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '193041babc604a37ac3d602710590d67', 'request_id': 'req-cb6e0afd-9ab1-40e2-b263-6c72a8f085a9'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:55.381276 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-cb6e0afd-9ab1-40e2-b263-6c72a8f085a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '193041babc604a37ac3d602710590d67', 'request_id': 'req-cb6e0afd-9ab1-40e2-b263-6c72a8f085a9'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:55.390574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-cb6e0afd-9ab1-40e2-b263-6c72a8f085a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '193041babc604a37ac3d602710590d67', 'request_id': 'req-cb6e0afd-9ab1-40e2-b263-6c72a8f085a9'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:55.407777 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-cb6e0afd-9ab1-40e2-b263-6c72a8f085a9 admin admin] The token cache is being invalidate because project 193041babc604a37ac3d602710590d67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:53:55.408076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-cb6e0afd-9ab1-40e2-b263-6c72a8f085a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:55.410939 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-cb6e0afd-9ab1-40e2-b263-6c72a8f085a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:53:55.423175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1262/2508] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:53:55 2022] DELETE /identity/v3/projects/193041babc604a37ac3d602710590d67 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:53:55.425541 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b6815-1637-4238-b37d-0737137b69c3 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:55.425541 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b6815-1637-4238-b37d-0737137b69c3 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:55.425887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b6815-1637-4238-b37d-0737137b69c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:55.452145 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-f14b6815-1637-4238-b37d-0737137b69c3 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:53:55.454826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f14b6815-1637-4238-b37d-0737137b69c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:55.531314 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1247/2509] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:53:55.539836 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477bb6fc-8911-4106-b1c3-7982fba85b87 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:55.540629 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477bb6fc-8911-4106-b1c3-7982fba85b87 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:55.540629 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477bb6fc-8911-4106-b1c3-7982fba85b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:55.560185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-477bb6fc-8911-4106-b1c3-7982fba85b87 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:53:55.561623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-477bb6fc-8911-4106-b1c3-7982fba85b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:55.570845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1263/2510] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:53:55.575711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b6815-1637-4238-b37d-0737137b69c3 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:55.585814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b6815-1637-4238-b37d-0737137b69c3 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:55.592468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc16d87-00b8-4d8e-8d68-5240aa358561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:55.593144 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc16d87-00b8-4d8e-8d68-5240aa358561 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:55.593429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc16d87-00b8-4d8e-8d68-5240aa358561 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:55.593708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc16d87-00b8-4d8e-8d68-5240aa358561 admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:55.594963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc16d87-00b8-4d8e-8d68-5240aa358561 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:53:55.595452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc16d87-00b8-4d8e-8d68-5240aa358561 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:55.597098 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc16d87-00b8-4d8e-8d68-5240aa358561 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:55.612547 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1248/2511] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:53:55.617232 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477bb6fc-8911-4106-b1c3-7982fba85b87 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:55.630865 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477bb6fc-8911-4106-b1c3-7982fba85b87 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:55.635934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2ad7d-5962-438d-ab4f-ac8dc8bbd634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:55.636596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2ad7d-5962-438d-ab4f-ac8dc8bbd634 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:55.637080 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2ad7d-5962-438d-ab4f-ac8dc8bbd634 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:55.637356 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2ad7d-5962-438d-ab4f-ac8dc8bbd634 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:55.638321 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2ad7d-5962-438d-ab4f-ac8dc8bbd634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:55.640286 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2ad7d-5962-438d-ab4f-ac8dc8bbd634 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:55.673952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1264/2512] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:53:55 2022] POST /identity/v3/projects => generated 357 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:53:55.678084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc16d87-00b8-4d8e-8d68-5240aa358561 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:55.693330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc16d87-00b8-4d8e-8d68-5240aa358561 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:55.699681 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:55.800161 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17a717-509c-4ac9-abb7-3cffadefeee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:55.800894 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17a717-509c-4ac9-abb7-3cffadefeee1 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:55.801182 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17a717-509c-4ac9-abb7-3cffadefeee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:55.801463 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17a717-509c-4ac9-abb7-3cffadefeee1 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:55.802187 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17a717-509c-4ac9-abb7-3cffadefeee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:55.804517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17a717-509c-4ac9-abb7-3cffadefeee1 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:55.834394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1249/2513] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:53:55 2022] POST /identity/v3/users => generated 279 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:53:55.838406 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2ad7d-5962-438d-ab4f-ac8dc8bbd634 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:55.850382 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2ad7d-5962-438d-ab4f-ac8dc8bbd634 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:55.856130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8f080-aef8-4c16-8bfa-7e7bf041b657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:55.859849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8f080-aef8-4c16-8bfa-7e7bf041b657 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:55.860219 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8f080-aef8-4c16-8bfa-7e7bf041b657 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:55.860562 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8f080-aef8-4c16-8bfa-7e7bf041b657 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:55.861923 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8f080-aef8-4c16-8bfa-7e7bf041b657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:55.863640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8f080-aef8-4c16-8bfa-7e7bf041b657 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:55.875548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1265/2514] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:53:55.879735 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17a717-509c-4ac9-abb7-3cffadefeee1 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:55.891890 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17a717-509c-4ac9-abb7-3cffadefeee1 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:55.898079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b51713-8e42-4cb7-82a2-26557e7a3b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:55.898997 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b51713-8e42-4cb7-82a2-26557e7a3b1d admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:55.899734 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b51713-8e42-4cb7-82a2-26557e7a3b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:55.900150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b51713-8e42-4cb7-82a2-26557e7a3b1d admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:55.901073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b51713-8e42-4cb7-82a2-26557e7a3b1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:55.903757 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b51713-8e42-4cb7-82a2-26557e7a3b1d admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:55.919168 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1250/2515] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:53:55.923620 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8f080-aef8-4c16-8bfa-7e7bf041b657 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:55.936280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8f080-aef8-4c16-8bfa-7e7bf041b657 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:55.953572 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc4922-2842-4dcd-993d-4c8c6fa0f403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:55.955032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc4922-2842-4dcd-993d-4c8c6fa0f403 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:55.955314 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc4922-2842-4dcd-993d-4c8c6fa0f403 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:55.955628 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc4922-2842-4dcd-993d-4c8c6fa0f403 admin admin] PATH_INFO: `/v3/projects/b27ad5a11e554359ab080b84e41a063d/users/ee9d50def1c545169a864051d2413c34/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:55.978540 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc4922-2842-4dcd-993d-4c8c6fa0f403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b27ad5a11e554359ab080b84e41a063d, user_id=ee9d50def1c545169a864051d2413c34, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:55.981276 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc4922-2842-4dcd-993d-4c8c6fa0f403 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:55.998825 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1266/2516] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:53:55 2022] PUT /identity/v3/projects/b27ad5a11e554359ab080b84e41a063d/users/ee9d50def1c545169a864051d2413c34/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:53:56.010267 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda6022-7c72-4454-b661-560451d78ce3 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:56.010563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda6022-7c72-4454-b661-560451d78ce3 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:56.010839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda6022-7c72-4454-b661-560451d78ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:56.040665 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-8fda6022-7c72-4454-b661-560451d78ce3 None None] MFA Rules not processed for user `ee9d50def1c545169a864051d2413c34`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:53:56.042211 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-8fda6022-7c72-4454-b661-560451d78ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:56.138343 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1251/2517] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:53:56 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:53:56.143447 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc4922-2842-4dcd-993d-4c8c6fa0f403 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:56.198178 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:56.224139 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc4922-2842-4dcd-993d-4c8c6fa0f403 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:56.229567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0a6ad-b85d-4ce3-9a9d-61340a7dc8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:56.230596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0a6ad-b85d-4ce3-9a9d-61340a7dc8e8 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:56.230924 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0a6ad-b85d-4ce3-9a9d-61340a7dc8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:56.231299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0a6ad-b85d-4ce3-9a9d-61340a7dc8e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:56.232122 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec0a6ad-b85d-4ce3-9a9d-61340a7dc8e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:56.234397 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec0a6ad-b85d-4ce3-9a9d-61340a7dc8e8 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:56.273249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1267/2518] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:53:56 2022] POST /identity/v3/projects => generated 357 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:53:56.277362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda6022-7c72-4454-b661-560451d78ce3 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:56.289541 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda6022-7c72-4454-b661-560451d78ce3 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:56.351229 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64697f-db66-43a3-963f-e4a8e5825a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:56.351944 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64697f-db66-43a3-963f-e4a8e5825a3a admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:56.352223 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64697f-db66-43a3-963f-e4a8e5825a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:56.352496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64697f-db66-43a3-963f-e4a8e5825a3a admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:56.353191 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b64697f-db66-43a3-963f-e4a8e5825a3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:56.355200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b64697f-db66-43a3-963f-e4a8e5825a3a admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:56.378156 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1252/2519] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:53:56 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:53:56.382097 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0a6ad-b85d-4ce3-9a9d-61340a7dc8e8 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:56.393564 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0a6ad-b85d-4ce3-9a9d-61340a7dc8e8 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:56.398704 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf1fcb-d1e0-4241-9fe3-c185487df710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:56.399399 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf1fcb-d1e0-4241-9fe3-c185487df710 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:56.399687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf1fcb-d1e0-4241-9fe3-c185487df710 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:56.399964 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf1fcb-d1e0-4241-9fe3-c185487df710 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:56.400688 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf1fcb-d1e0-4241-9fe3-c185487df710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:56.402287 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf1fcb-d1e0-4241-9fe3-c185487df710 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:56.416573 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1268/2520] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:53:56.420997 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64697f-db66-43a3-963f-e4a8e5825a3a admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:56.431175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64697f-db66-43a3-963f-e4a8e5825a3a admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:56.436057 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e7364-27e4-4b67-af56-9621a124f37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:56.436933 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e7364-27e4-4b67-af56-9621a124f37b admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:56.437221 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e7364-27e4-4b67-af56-9621a124f37b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:56.437515 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e7364-27e4-4b67-af56-9621a124f37b admin admin] PATH_INFO: `/v3/projects/bada479d3db34034826d35fb79913b29/users/3ea70d6d3c5c431b8068d907ce0fec17/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:56.444615 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:56.501382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647e7364-27e4-4b67-af56-9621a124f37b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bada479d3db34034826d35fb79913b29, user_id=3ea70d6d3c5c431b8068d907ce0fec17, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:56.504213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647e7364-27e4-4b67-af56-9621a124f37b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:56.526280 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1253/2521] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:53:56 2022] PUT /identity/v3/projects/bada479d3db34034826d35fb79913b29/users/3ea70d6d3c5c431b8068d907ce0fec17/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:53:56.530653 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf1fcb-d1e0-4241-9fe3-c185487df710 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:56.545489 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf1fcb-d1e0-4241-9fe3-c185487df710 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:56.615258 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326a0f7-547f-483d-b256-d972d90e92fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:56.616140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326a0f7-547f-483d-b256-d972d90e92fb admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:56.616241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326a0f7-547f-483d-b256-d972d90e92fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:56.616530 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326a0f7-547f-483d-b256-d972d90e92fb admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:56.617386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326a0f7-547f-483d-b256-d972d90e92fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:56.619174 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326a0f7-547f-483d-b256-d972d90e92fb admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:56.630817 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1269/2522] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:53:56.635307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e7364-27e4-4b67-af56-9621a124f37b admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:56.657659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e7364-27e4-4b67-af56-9621a124f37b admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:56.669694 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1e424-d5b7-4e0c-9d8d-2c3da7bae250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:56.670742 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1e424-d5b7-4e0c-9d8d-2c3da7bae250 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:56.671133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1e424-d5b7-4e0c-9d8d-2c3da7bae250 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:56.671554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1e424-d5b7-4e0c-9d8d-2c3da7bae250 admin admin] PATH_INFO: `/v3/projects/bada479d3db34034826d35fb79913b29/users/3ea70d6d3c5c431b8068d907ce0fec17/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:56.674294 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1e424-d5b7-4e0c-9d8d-2c3da7bae250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bada479d3db34034826d35fb79913b29, user_id=3ea70d6d3c5c431b8068d907ce0fec17, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:56.677586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1e424-d5b7-4e0c-9d8d-2c3da7bae250 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:56.690955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1254/2523] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:53:56 2022] PUT /identity/v3/projects/bada479d3db34034826d35fb79913b29/users/3ea70d6d3c5c431b8068d907ce0fec17/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:53:56.701235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f5421-ca46-4082-96f6-1f4f70e01da2 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:56.701826 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f5421-ca46-4082-96f6-1f4f70e01da2 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:56.702113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f5421-ca46-4082-96f6-1f4f70e01da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:56.733167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-fa7f5421-ca46-4082-96f6-1f4f70e01da2 None None] MFA Rules not processed for user `3ea70d6d3c5c431b8068d907ce0fec17`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:53:56.734378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-fa7f5421-ca46-4082-96f6-1f4f70e01da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:56.832495 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1270/2524] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:53:56 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:53:56.845557 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1e424-d5b7-4e0c-9d8d-2c3da7bae250 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:56.848470 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-31a1e424-d5b7-4e0c-9d8d-2c3da7bae250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:53:56.914036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1e424-d5b7-4e0c-9d8d-2c3da7bae250 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:56.917911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14e8a4-dd26-414a-8fe1-10aedbb79857 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:56.918638 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14e8a4-dd26-414a-8fe1-10aedbb79857 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:56.918994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14e8a4-dd26-414a-8fe1-10aedbb79857 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:56.919344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14e8a4-dd26-414a-8fe1-10aedbb79857 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:56.931666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14e8a4-dd26-414a-8fe1-10aedbb79857 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:56.932934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14e8a4-dd26-414a-8fe1-10aedbb79857 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:56.945294 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1255/2525] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:53:56 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:53:57.020732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f5421-ca46-4082-96f6-1f4f70e01da2 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:53:57.031041 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f5421-ca46-4082-96f6-1f4f70e01da2 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:53:57.036062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ee332-0deb-4d83-a67f-92016487873e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:53:57.036715 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ee332-0deb-4d83-a67f-92016487873e service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:53:57.037143 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ee332-0deb-4d83-a67f-92016487873e service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:53:57.037422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ee332-0deb-4d83-a67f-92016487873e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:53:57.077743 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:53:57.108869 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ee332-0deb-4d83-a67f-92016487873e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:53:57.110965 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ee332-0deb-4d83-a67f-92016487873e service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:53:57.120605 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1271/2526] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:53:57 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:54:06.939673 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=38,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:07.116586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=12 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:13.204361 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14e8a4-dd26-414a-8fe1-10aedbb79857 service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:13.267017 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14e8a4-dd26-414a-8fe1-10aedbb79857 service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:13.271675 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45187792-9bcd-403b-b137-1e17795a60c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:13.272547 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45187792-9bcd-403b-b137-1e17795a60c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:13.272830 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45187792-9bcd-403b-b137-1e17795a60c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:13.273105 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45187792-9bcd-403b-b137-1e17795a60c5 admin admin] PATH_INFO: `/v3/users/ee9d50def1c545169a864051d2413c34` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:13.274300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45187792-9bcd-403b-b137-1e17795a60c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee9d50def1c545169a864051d2413c34)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:13.276244 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45187792-9bcd-403b-b137-1e17795a60c5 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:13.323423 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-45187792-9bcd-403b-b137-1e17795a60c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee9d50def1c545169a864051d2413c34'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:13.335033 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-45187792-9bcd-403b-b137-1e17795a60c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee9d50def1c545169a864051d2413c34'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:13.342158 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-45187792-9bcd-403b-b137-1e17795a60c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee9d50def1c545169a864051d2413c34'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:13.342800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-45187792-9bcd-403b-b137-1e17795a60c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee9d50def1c545169a864051d2413c34'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:13.349627 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-45187792-9bcd-403b-b137-1e17795a60c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee9d50def1c545169a864051d2413c34'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:13.355474 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1256/2527] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:54:13 2022] DELETE /identity/v3/users/ee9d50def1c545169a864051d2413c34 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:54:13.360013 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89c544-44b1-4bb8-a6ce-85b6fd3c6340 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:13.360518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89c544-44b1-4bb8-a6ce-85b6fd3c6340 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:13.360617 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f89c544-44b1-4bb8-a6ce-85b6fd3c6340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:13.378622 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-8f89c544-44b1-4bb8-a6ce-85b6fd3c6340 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:54:13.379865 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-8f89c544-44b1-4bb8-a6ce-85b6fd3c6340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:13.442827 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1272/2528] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:54:13.452573 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45187792-9bcd-403b-b137-1e17795a60c5 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:13.453988 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-45187792-9bcd-403b-b137-1e17795a60c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:13.510694 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45187792-9bcd-403b-b137-1e17795a60c5 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:13.514711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ed735-1716-498f-91c3-890903292a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:13.515386 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ed735-1716-498f-91c3-890903292a86 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:13.515676 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ed735-1716-498f-91c3-890903292a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:13.515955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ed735-1716-498f-91c3-890903292a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:13.525490 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ed735-1716-498f-91c3-890903292a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:13.526932 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ed735-1716-498f-91c3-890903292a86 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:13.537270 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1257/2529] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:54:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:54:13.801733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89c544-44b1-4bb8-a6ce-85b6fd3c6340 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:13.803230 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-8f89c544-44b1-4bb8-a6ce-85b6fd3c6340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:13.817526 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f89c544-44b1-4bb8-a6ce-85b6fd3c6340 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:13.822376 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2333b9-ec24-4fcb-9bfc-20d35366b35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:13.823245 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2333b9-ec24-4fcb-9bfc-20d35366b35e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:13.823567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2333b9-ec24-4fcb-9bfc-20d35366b35e admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:13.823844 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2333b9-ec24-4fcb-9bfc-20d35366b35e admin admin] PATH_INFO: `/v3/projects/b27ad5a11e554359ab080b84e41a063d` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:13.825386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2333b9-ec24-4fcb-9bfc-20d35366b35e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b27ad5a11e554359ab080b84e41a063d)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:13.827473 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2333b9-ec24-4fcb-9bfc-20d35366b35e admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:13.875697 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6a2333b9-ec24-4fcb-9bfc-20d35366b35e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b27ad5a11e554359ab080b84e41a063d', 'request_id': 'req-6a2333b9-ec24-4fcb-9bfc-20d35366b35e'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:13.881981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6a2333b9-ec24-4fcb-9bfc-20d35366b35e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b27ad5a11e554359ab080b84e41a063d', 'request_id': 'req-6a2333b9-ec24-4fcb-9bfc-20d35366b35e'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:13.890535 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6a2333b9-ec24-4fcb-9bfc-20d35366b35e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b27ad5a11e554359ab080b84e41a063d', 'request_id': 'req-6a2333b9-ec24-4fcb-9bfc-20d35366b35e'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:13.901784 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6a2333b9-ec24-4fcb-9bfc-20d35366b35e admin admin] The token cache is being invalidate because project b27ad5a11e554359ab080b84e41a063d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:54:13.902076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6a2333b9-ec24-4fcb-9bfc-20d35366b35e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:13.902711 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6a2333b9-ec24-4fcb-9bfc-20d35366b35e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:13.904290 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1273/2530] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:54:13 2022] DELETE /identity/v3/projects/b27ad5a11e554359ab080b84e41a063d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:54:13.909000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ed735-1716-498f-91c3-890903292a86 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:13.919203 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ed735-1716-498f-91c3-890903292a86 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:13.937659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:13.979053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6058c78-301b-415b-b3a7-2d7c8fe6886b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:13.979887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6058c78-301b-415b-b3a7-2d7c8fe6886b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:13.980174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6058c78-301b-415b-b3a7-2d7c8fe6886b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:13.980454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6058c78-301b-415b-b3a7-2d7c8fe6886b admin admin] PATH_INFO: `/v3/users/3ea70d6d3c5c431b8068d907ce0fec17` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:13.981648 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6058c78-301b-415b-b3a7-2d7c8fe6886b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ea70d6d3c5c431b8068d907ce0fec17)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:13.983976 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6058c78-301b-415b-b3a7-2d7c8fe6886b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:14.029330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a6058c78-301b-415b-b3a7-2d7c8fe6886b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ea70d6d3c5c431b8068d907ce0fec17'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:14.041621 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a6058c78-301b-415b-b3a7-2d7c8fe6886b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ea70d6d3c5c431b8068d907ce0fec17'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:14.048963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a6058c78-301b-415b-b3a7-2d7c8fe6886b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ea70d6d3c5c431b8068d907ce0fec17'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:14.049928 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a6058c78-301b-415b-b3a7-2d7c8fe6886b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ea70d6d3c5c431b8068d907ce0fec17'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:14.056874 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-a6058c78-301b-415b-b3a7-2d7c8fe6886b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ea70d6d3c5c431b8068d907ce0fec17'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:14.062709 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1258/2531] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:54:13 2022] DELETE /identity/v3/users/3ea70d6d3c5c431b8068d907ce0fec17 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:54:14.336398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2333b9-ec24-4fcb-9bfc-20d35366b35e admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:14.337812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-6a2333b9-ec24-4fcb-9bfc-20d35366b35e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:14.351110 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2333b9-ec24-4fcb-9bfc-20d35366b35e admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:14.409607 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f6fc7-494d-40ca-b4b9-95a10e5da642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:14.410465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f6fc7-494d-40ca-b4b9-95a10e5da642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:14.410774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f6fc7-494d-40ca-b4b9-95a10e5da642 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:14.411049 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f6fc7-494d-40ca-b4b9-95a10e5da642 admin admin] PATH_INFO: `/v3/projects/bada479d3db34034826d35fb79913b29` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:14.412250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f6fc7-494d-40ca-b4b9-95a10e5da642 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bada479d3db34034826d35fb79913b29)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:14.414146 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f6fc7-494d-40ca-b4b9-95a10e5da642 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:14.446011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:14.464748 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-807f6fc7-494d-40ca-b4b9-95a10e5da642 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bada479d3db34034826d35fb79913b29', 'request_id': 'req-807f6fc7-494d-40ca-b4b9-95a10e5da642'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:14.470781 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-807f6fc7-494d-40ca-b4b9-95a10e5da642 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bada479d3db34034826d35fb79913b29', 'request_id': 'req-807f6fc7-494d-40ca-b4b9-95a10e5da642'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:14.478045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-807f6fc7-494d-40ca-b4b9-95a10e5da642 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bada479d3db34034826d35fb79913b29', 'request_id': 'req-807f6fc7-494d-40ca-b4b9-95a10e5da642'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:14.488607 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-807f6fc7-494d-40ca-b4b9-95a10e5da642 admin admin] The token cache is being invalidate because project bada479d3db34034826d35fb79913b29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:54:14.488894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-807f6fc7-494d-40ca-b4b9-95a10e5da642 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:14.489518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-807f6fc7-494d-40ca-b4b9-95a10e5da642 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:14.491056 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1274/2532] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:54:14 2022] DELETE /identity/v3/projects/bada479d3db34034826d35fb79913b29 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:54:14.496002 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2a9b5-5374-478b-b5a7-af2d6d7a333c None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:14.496485 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2a9b5-5374-478b-b5a7-af2d6d7a333c None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:14.496580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2a9b5-5374-478b-b5a7-af2d6d7a333c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:14.515711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-51b2a9b5-5374-478b-b5a7-af2d6d7a333c None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:54:14.516836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-51b2a9b5-5374-478b-b5a7-af2d6d7a333c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:14.576872 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1259/2533] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:54:14.584800 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3eafd-1c79-40e4-8511-acb8f6ec10ff None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:14.585087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3eafd-1c79-40e4-8511-acb8f6ec10ff None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:14.585366 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3eafd-1c79-40e4-8511-acb8f6ec10ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:14.603984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-26c3eafd-1c79-40e4-8511-acb8f6ec10ff None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:54:14.605180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-26c3eafd-1c79-40e4-8511-acb8f6ec10ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:14.614786 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1275/2534] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:54:14.618973 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2a9b5-5374-478b-b5a7-af2d6d7a333c None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:14.629447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2a9b5-5374-478b-b5a7-af2d6d7a333c None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:14.634210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90ff7f-b200-431b-886e-27617433054a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:14.634865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90ff7f-b200-431b-886e-27617433054a admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:14.635151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90ff7f-b200-431b-886e-27617433054a admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:14.635464 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90ff7f-b200-431b-886e-27617433054a admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:14.636193 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa90ff7f-b200-431b-886e-27617433054a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:54:14.636500 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa90ff7f-b200-431b-886e-27617433054a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:14.638373 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa90ff7f-b200-431b-886e-27617433054a admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:14.651025 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1260/2535] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:54:14.654602 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3eafd-1c79-40e4-8511-acb8f6ec10ff None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:14.664333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3eafd-1c79-40e4-8511-acb8f6ec10ff None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:14.669109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d88890-e80f-4b8e-8617-1fd88161516f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:14.669798 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d88890-e80f-4b8e-8617-1fd88161516f admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:14.670087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d88890-e80f-4b8e-8617-1fd88161516f admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:14.670466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d88890-e80f-4b8e-8617-1fd88161516f admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:14.671063 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d88890-e80f-4b8e-8617-1fd88161516f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:14.673078 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d88890-e80f-4b8e-8617-1fd88161516f admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:14.700502 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1276/2536] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:54:14 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:54:14.706300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90ff7f-b200-431b-886e-27617433054a admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:14.716466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90ff7f-b200-431b-886e-27617433054a admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:14.745961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:14.778078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc2722-3b87-4fed-a3a4-b685f8223ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:14.778679 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc2722-3b87-4fed-a3a4-b685f8223ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:14.778994 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc2722-3b87-4fed-a3a4-b685f8223ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:14.779276 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc2722-3b87-4fed-a3a4-b685f8223ec5 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:14.780020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc2722-3b87-4fed-a3a4-b685f8223ec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:14.781950 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc2722-3b87-4fed-a3a4-b685f8223ec5 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:14.801707 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1261/2537] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:54:14 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:54:14.808065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d88890-e80f-4b8e-8617-1fd88161516f admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:14.824612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d88890-e80f-4b8e-8617-1fd88161516f admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:14.829286 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55c403-6bda-4712-bba7-646bf4a30c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:14.829956 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55c403-6bda-4712-bba7-646bf4a30c2b admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:14.830389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55c403-6bda-4712-bba7-646bf4a30c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:14.830669 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55c403-6bda-4712-bba7-646bf4a30c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:14.831421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55c403-6bda-4712-bba7-646bf4a30c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:14.833017 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55c403-6bda-4712-bba7-646bf4a30c2b admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:14.845000 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1277/2538] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:54:14.849119 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc2722-3b87-4fed-a3a4-b685f8223ec5 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:14.859264 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc2722-3b87-4fed-a3a4-b685f8223ec5 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:14.864505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9724b-2bf8-480c-a758-b2ef0b4f8bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:14.865156 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9724b-2bf8-480c-a758-b2ef0b4f8bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:14.865441 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9724b-2bf8-480c-a758-b2ef0b4f8bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:14.865748 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9724b-2bf8-480c-a758-b2ef0b4f8bf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:14.866466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9724b-2bf8-480c-a758-b2ef0b4f8bf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:14.868102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9724b-2bf8-480c-a758-b2ef0b4f8bf2 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:14.879146 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1262/2539] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:54:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:54:14.883613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55c403-6bda-4712-bba7-646bf4a30c2b admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:14.894419 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55c403-6bda-4712-bba7-646bf4a30c2b admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:14.899080 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4edb9-8155-4ca5-8440-725701afbb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:14.899998 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4edb9-8155-4ca5-8440-725701afbb8b admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:14.900278 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4edb9-8155-4ca5-8440-725701afbb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:14.900752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4edb9-8155-4ca5-8440-725701afbb8b admin admin] PATH_INFO: `/v3/projects/3edc7ae699a047cdbc75990e6ee52ba3/users/de282c74785d4a2db4b7cf5c9b85f63b/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:14.916300 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4edb9-8155-4ca5-8440-725701afbb8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3edc7ae699a047cdbc75990e6ee52ba3, user_id=de282c74785d4a2db4b7cf5c9b85f63b, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:14.918943 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4edb9-8155-4ca5-8440-725701afbb8b admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:14.933569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1278/2540] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:54:14 2022] PUT /identity/v3/projects/3edc7ae699a047cdbc75990e6ee52ba3/users/de282c74785d4a2db4b7cf5c9b85f63b/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:54:14.945502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b713f-ab70-49a0-9bbb-9ca2e1a2bdce None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:14.945502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b713f-ab70-49a0-9bbb-9ca2e1a2bdce None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:14.945502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b713f-ab70-49a0-9bbb-9ca2e1a2bdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:14.974457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-de9b713f-ab70-49a0-9bbb-9ca2e1a2bdce None None] MFA Rules not processed for user `de282c74785d4a2db4b7cf5c9b85f63b`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:54:14.975713 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-de9b713f-ab70-49a0-9bbb-9ca2e1a2bdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:15.045551 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1263/2541] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:54:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:54:15.051271 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4edb9-8155-4ca5-8440-725701afbb8b admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:15.113374 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4edb9-8155-4ca5-8440-725701afbb8b admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:15.117991 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d7ab8-9396-4aca-8af9-c632e97df014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:15.118645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d7ab8-9396-4aca-8af9-c632e97df014 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:15.118926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d7ab8-9396-4aca-8af9-c632e97df014 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:15.119209 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d7ab8-9396-4aca-8af9-c632e97df014 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:15.119937 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d7ab8-9396-4aca-8af9-c632e97df014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:15.121872 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d7ab8-9396-4aca-8af9-c632e97df014 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:15.148064 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1279/2542] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:54:15 2022] POST /identity/v3/projects => generated 393 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:54:15.152372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b713f-ab70-49a0-9bbb-9ca2e1a2bdce None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:15.164872 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b713f-ab70-49a0-9bbb-9ca2e1a2bdce None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:15.220078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1a415-8554-444c-b161-dafc2cd6bf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:15.221251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1a415-8554-444c-b161-dafc2cd6bf5d admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:15.221538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1a415-8554-444c-b161-dafc2cd6bf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:15.221819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1a415-8554-444c-b161-dafc2cd6bf5d admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:15.222522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1a415-8554-444c-b161-dafc2cd6bf5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:15.224541 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1a415-8554-444c-b161-dafc2cd6bf5d admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:15.244584 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1264/2543] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:54:15 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:54:15.248605 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d7ab8-9396-4aca-8af9-c632e97df014 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:15.259123 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d7ab8-9396-4aca-8af9-c632e97df014 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:15.264019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b636373-3829-491f-934f-6fdd6db5dc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:15.264667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b636373-3829-491f-934f-6fdd6db5dc90 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:15.264950 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b636373-3829-491f-934f-6fdd6db5dc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:15.265225 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b636373-3829-491f-934f-6fdd6db5dc90 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:15.265942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b636373-3829-491f-934f-6fdd6db5dc90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:15.270050 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b636373-3829-491f-934f-6fdd6db5dc90 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:15.280732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1280/2544] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:54:15.285828 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1a415-8554-444c-b161-dafc2cd6bf5d admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:15.296074 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1a415-8554-444c-b161-dafc2cd6bf5d admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:15.300679 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61bb29-601c-4e57-a627-d0220326c7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:15.301553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61bb29-601c-4e57-a627-d0220326c7fa admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:15.301840 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61bb29-601c-4e57-a627-d0220326c7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:15.302121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61bb29-601c-4e57-a627-d0220326c7fa admin admin] PATH_INFO: `/v3/projects/d4fc0666929441b79d64a971bbc34a98/users/42b4a4bce6dd4747a521a56028177c63/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:15.316523 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61bb29-601c-4e57-a627-d0220326c7fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4fc0666929441b79d64a971bbc34a98, user_id=42b4a4bce6dd4747a521a56028177c63, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:15.319158 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61bb29-601c-4e57-a627-d0220326c7fa admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:15.333153 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1265/2545] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:54:15 2022] PUT /identity/v3/projects/d4fc0666929441b79d64a971bbc34a98/users/42b4a4bce6dd4747a521a56028177c63/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:54:15.337363 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b636373-3829-491f-934f-6fdd6db5dc90 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:15.347971 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b636373-3829-491f-934f-6fdd6db5dc90 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:15.367423 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:15.406667 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb9235-c8a9-4a33-ae56-24fea1c1ae5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:15.407333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb9235-c8a9-4a33-ae56-24fea1c1ae5b admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:15.407654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb9235-c8a9-4a33-ae56-24fea1c1ae5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:15.408530 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb9235-c8a9-4a33-ae56-24fea1c1ae5b admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:15.409282 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb9235-c8a9-4a33-ae56-24fea1c1ae5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:15.410879 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb9235-c8a9-4a33-ae56-24fea1c1ae5b admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:15.420685 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1281/2546] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:54:15.424849 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61bb29-601c-4e57-a627-d0220326c7fa admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:15.430380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:15.437040 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61bb29-601c-4e57-a627-d0220326c7fa admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:15.442222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38adb38-0d04-441c-a16f-d566a51df024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:15.443127 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38adb38-0d04-441c-a16f-d566a51df024 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:15.443680 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38adb38-0d04-441c-a16f-d566a51df024 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:15.444234 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38adb38-0d04-441c-a16f-d566a51df024 admin admin] PATH_INFO: `/v3/projects/d4fc0666929441b79d64a971bbc34a98/users/42b4a4bce6dd4747a521a56028177c63/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:15.446298 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38adb38-0d04-441c-a16f-d566a51df024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4fc0666929441b79d64a971bbc34a98, user_id=42b4a4bce6dd4747a521a56028177c63, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:15.449066 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38adb38-0d04-441c-a16f-d566a51df024 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:15.460226 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1266/2547] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:54:15 2022] PUT /identity/v3/projects/d4fc0666929441b79d64a971bbc34a98/users/42b4a4bce6dd4747a521a56028177c63/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:54:15.470391 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcd2c0-1737-4638-a883-8cb3d7c02572 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:15.471241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcd2c0-1737-4638-a883-8cb3d7c02572 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:15.471568 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcd2c0-1737-4638-a883-8cb3d7c02572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:15.497828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-26bcd2c0-1737-4638-a883-8cb3d7c02572 None None] MFA Rules not processed for user `42b4a4bce6dd4747a521a56028177c63`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:54:15.499056 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-26bcd2c0-1737-4638-a883-8cb3d7c02572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:15.567453 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1282/2548] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:54:15 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:54:15.578380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38adb38-0d04-441c-a16f-d566a51df024 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:15.581155 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a38adb38-0d04-441c-a16f-d566a51df024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:15.638984 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38adb38-0d04-441c-a16f-d566a51df024 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:15.643277 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449276e-3c72-4127-a43e-826b4f6c12e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:15.643939 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449276e-3c72-4127-a43e-826b4f6c12e0 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:15.644221 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449276e-3c72-4127-a43e-826b4f6c12e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:15.644492 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449276e-3c72-4127-a43e-826b4f6c12e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:15.653577 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449276e-3c72-4127-a43e-826b4f6c12e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:15.655030 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449276e-3c72-4127-a43e-826b4f6c12e0 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:15.665103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1267/2549] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:54:15 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:54:15.710161 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcd2c0-1737-4638-a883-8cb3d7c02572 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:15.712151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-26bcd2c0-1737-4638-a883-8cb3d7c02572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:15.724126 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcd2c0-1737-4638-a883-8cb3d7c02572 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:15.728100 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310f7f7-68ed-4e7d-a97d-4a69579e8763 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:15.728745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310f7f7-68ed-4e7d-a97d-4a69579e8763 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:15.729021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310f7f7-68ed-4e7d-a97d-4a69579e8763 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:15.729365 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310f7f7-68ed-4e7d-a97d-4a69579e8763 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:15.788090 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310f7f7-68ed-4e7d-a97d-4a69579e8763 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:15.789525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310f7f7-68ed-4e7d-a97d-4a69579e8763 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:15.798685 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1283/2550] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:54:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:54:20.117646 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449276e-3c72-4127-a43e-826b4f6c12e0 service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:20.177854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449276e-3c72-4127-a43e-826b4f6c12e0 service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:20.182317 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571f37e-dd54-40fa-8c8d-508211f76657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:20.183172 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571f37e-dd54-40fa-8c8d-508211f76657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:20.183493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571f37e-dd54-40fa-8c8d-508211f76657 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:20.183770 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571f37e-dd54-40fa-8c8d-508211f76657 admin admin] PATH_INFO: `/v3/users/de282c74785d4a2db4b7cf5c9b85f63b` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:20.184943 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d571f37e-dd54-40fa-8c8d-508211f76657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de282c74785d4a2db4b7cf5c9b85f63b)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:20.186922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d571f37e-dd54-40fa-8c8d-508211f76657 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:20.234486 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d571f37e-dd54-40fa-8c8d-508211f76657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de282c74785d4a2db4b7cf5c9b85f63b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:20.246084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d571f37e-dd54-40fa-8c8d-508211f76657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de282c74785d4a2db4b7cf5c9b85f63b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:20.254135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d571f37e-dd54-40fa-8c8d-508211f76657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de282c74785d4a2db4b7cf5c9b85f63b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:20.254819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d571f37e-dd54-40fa-8c8d-508211f76657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de282c74785d4a2db4b7cf5c9b85f63b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:20.262122 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-d571f37e-dd54-40fa-8c8d-508211f76657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de282c74785d4a2db4b7cf5c9b85f63b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:20.268977 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1268/2551] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:54:20 2022] DELETE /identity/v3/users/de282c74785d4a2db4b7cf5c9b85f63b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:54:20.273437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7192bfb-76c4-4246-b2eb-397e663011a3 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:20.273914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7192bfb-76c4-4246-b2eb-397e663011a3 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:20.274053 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7192bfb-76c4-4246-b2eb-397e663011a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:20.290942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d7192bfb-76c4-4246-b2eb-397e663011a3 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:54:20.292158 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d7192bfb-76c4-4246-b2eb-397e663011a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:20.331644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:20.353095 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1284/2552] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:54:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:54:20.362788 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571f37e-dd54-40fa-8c8d-508211f76657 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:20.364220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d571f37e-dd54-40fa-8c8d-508211f76657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:20.378163 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:20.423534 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571f37e-dd54-40fa-8c8d-508211f76657 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:20.427422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d358b0-5dab-4dba-b9d1-49687b2ba724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:20.428091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d358b0-5dab-4dba-b9d1-49687b2ba724 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:20.428379 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d358b0-5dab-4dba-b9d1-49687b2ba724 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:20.428651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d358b0-5dab-4dba-b9d1-49687b2ba724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:20.437977 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d358b0-5dab-4dba-b9d1-49687b2ba724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:20.440009 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d358b0-5dab-4dba-b9d1-49687b2ba724 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:20.450598 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1269/2553] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:54:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:54:20.715915 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7192bfb-76c4-4246-b2eb-397e663011a3 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:20.717386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d7192bfb-76c4-4246-b2eb-397e663011a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:20.729367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7192bfb-76c4-4246-b2eb-397e663011a3 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:20.734109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3d528-7dfd-4183-951a-9d59c200e92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:20.734960 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3d528-7dfd-4183-951a-9d59c200e92d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:20.735242 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3d528-7dfd-4183-951a-9d59c200e92d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:20.735590 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3d528-7dfd-4183-951a-9d59c200e92d admin admin] PATH_INFO: `/v3/projects/3edc7ae699a047cdbc75990e6ee52ba3` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:20.737320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3d528-7dfd-4183-951a-9d59c200e92d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3edc7ae699a047cdbc75990e6ee52ba3)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:20.739224 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3d528-7dfd-4183-951a-9d59c200e92d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:20.789737 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c4e3d528-7dfd-4183-951a-9d59c200e92d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3edc7ae699a047cdbc75990e6ee52ba3', 'request_id': 'req-c4e3d528-7dfd-4183-951a-9d59c200e92d'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:20.795868 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c4e3d528-7dfd-4183-951a-9d59c200e92d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3edc7ae699a047cdbc75990e6ee52ba3', 'request_id': 'req-c4e3d528-7dfd-4183-951a-9d59c200e92d'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:20.803781 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c4e3d528-7dfd-4183-951a-9d59c200e92d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3edc7ae699a047cdbc75990e6ee52ba3', 'request_id': 'req-c4e3d528-7dfd-4183-951a-9d59c200e92d'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:20.815181 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c4e3d528-7dfd-4183-951a-9d59c200e92d admin admin] The token cache is being invalidate because project 3edc7ae699a047cdbc75990e6ee52ba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:54:20.815517 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c4e3d528-7dfd-4183-951a-9d59c200e92d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:20.816147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-c4e3d528-7dfd-4183-951a-9d59c200e92d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:20.817690 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1285/2554] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:54:20 2022] DELETE /identity/v3/projects/3edc7ae699a047cdbc75990e6ee52ba3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:54:20.822180 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d358b0-5dab-4dba-b9d1-49687b2ba724 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:20.831883 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d358b0-5dab-4dba-b9d1-49687b2ba724 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:20.886308 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4a884-211f-4dfe-977a-f77016122929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:20.887178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4a884-211f-4dfe-977a-f77016122929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:20.887745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4a884-211f-4dfe-977a-f77016122929 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:20.888023 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4a884-211f-4dfe-977a-f77016122929 admin admin] PATH_INFO: `/v3/users/42b4a4bce6dd4747a521a56028177c63` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:20.889201 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4a884-211f-4dfe-977a-f77016122929 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42b4a4bce6dd4747a521a56028177c63)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:20.891101 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4a884-211f-4dfe-977a-f77016122929 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:20.944716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-85d4a884-211f-4dfe-977a-f77016122929 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42b4a4bce6dd4747a521a56028177c63'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:20.956963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-85d4a884-211f-4dfe-977a-f77016122929 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42b4a4bce6dd4747a521a56028177c63'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:20.964625 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-85d4a884-211f-4dfe-977a-f77016122929 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42b4a4bce6dd4747a521a56028177c63'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:20.965499 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-85d4a884-211f-4dfe-977a-f77016122929 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42b4a4bce6dd4747a521a56028177c63'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:20.972875 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-85d4a884-211f-4dfe-977a-f77016122929 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42b4a4bce6dd4747a521a56028177c63'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:20.978785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1270/2555] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:54:20 2022] DELETE /identity/v3/users/42b4a4bce6dd4747a521a56028177c63 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:54:21.248841 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3d528-7dfd-4183-951a-9d59c200e92d admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:21.250312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-c4e3d528-7dfd-4183-951a-9d59c200e92d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:21.263691 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3d528-7dfd-4183-951a-9d59c200e92d admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:21.321680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4bc3fb-4778-4676-94c5-59d2056a9155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:21.322536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bc3fb-4778-4676-94c5-59d2056a9155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:21.322816 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bc3fb-4778-4676-94c5-59d2056a9155 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:21.323090 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4bc3fb-4778-4676-94c5-59d2056a9155 admin admin] PATH_INFO: `/v3/projects/d4fc0666929441b79d64a971bbc34a98` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:21.324298 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4bc3fb-4778-4676-94c5-59d2056a9155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4fc0666929441b79d64a971bbc34a98)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:21.326287 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4bc3fb-4778-4676-94c5-59d2056a9155 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:21.373634 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3b4bc3fb-4778-4676-94c5-59d2056a9155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4fc0666929441b79d64a971bbc34a98', 'request_id': 'req-3b4bc3fb-4778-4676-94c5-59d2056a9155'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:21.380355 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3b4bc3fb-4778-4676-94c5-59d2056a9155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4fc0666929441b79d64a971bbc34a98', 'request_id': 'req-3b4bc3fb-4778-4676-94c5-59d2056a9155'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:21.388391 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3b4bc3fb-4778-4676-94c5-59d2056a9155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4fc0666929441b79d64a971bbc34a98', 'request_id': 'req-3b4bc3fb-4778-4676-94c5-59d2056a9155'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:21.395229 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:21.400089 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3b4bc3fb-4778-4676-94c5-59d2056a9155 admin admin] The token cache is being invalidate because project d4fc0666929441b79d64a971bbc34a98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:54:21.400367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3b4bc3fb-4778-4676-94c5-59d2056a9155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:21.402113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3b4bc3fb-4778-4676-94c5-59d2056a9155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:21.403934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1286/2556] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:54:21 2022] DELETE /identity/v3/projects/d4fc0666929441b79d64a971bbc34a98 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:54:21.409630 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b4e2-5a1b-4490-a1c6-f3882c33be5f None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:21.411386 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b4e2-5a1b-4490-a1c6-f3882c33be5f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:21.411386 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b4e2-5a1b-4490-a1c6-f3882c33be5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:21.429368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-0e69b4e2-5a1b-4490-a1c6-f3882c33be5f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:54:21.430579 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-0e69b4e2-5a1b-4490-a1c6-f3882c33be5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:21.459119 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:21.497218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1271/2557] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:54:21.506228 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d2e64-e95f-4fdf-9d3a-600dbe608fcb None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:21.506513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d2e64-e95f-4fdf-9d3a-600dbe608fcb None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:21.506792 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d2e64-e95f-4fdf-9d3a-600dbe608fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:21.526903 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d12d2e64-e95f-4fdf-9d3a-600dbe608fcb None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:54:21.528135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d12d2e64-e95f-4fdf-9d3a-600dbe608fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:21.536680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1287/2558] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:54:21.541087 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69b4e2-5a1b-4490-a1c6-f3882c33be5f None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:21.550959 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69b4e2-5a1b-4490-a1c6-f3882c33be5f None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:21.555640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501be98d-a298-4d1c-ad94-884be527c29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:21.556295 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501be98d-a298-4d1c-ad94-884be527c29c admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:21.556581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501be98d-a298-4d1c-ad94-884be527c29c admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:21.556862 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501be98d-a298-4d1c-ad94-884be527c29c admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:21.557617 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501be98d-a298-4d1c-ad94-884be527c29c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:54:21.557924 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501be98d-a298-4d1c-ad94-884be527c29c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:21.559551 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501be98d-a298-4d1c-ad94-884be527c29c admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:21.571232 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1272/2559] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:54:21.575577 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d2e64-e95f-4fdf-9d3a-600dbe608fcb None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:21.585248 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d2e64-e95f-4fdf-9d3a-600dbe608fcb None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:21.589863 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac314c-3a78-4a1a-a855-6249448ee5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:21.590509 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac314c-3a78-4a1a-a855-6249448ee5f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:21.590790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac314c-3a78-4a1a-a855-6249448ee5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:21.591067 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac314c-3a78-4a1a-a855-6249448ee5f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:21.591891 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac314c-3a78-4a1a-a855-6249448ee5f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:21.593797 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac314c-3a78-4a1a-a855-6249448ee5f5 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:21.619612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1288/2560] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:54:21 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:54:21.623834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501be98d-a298-4d1c-ad94-884be527c29c admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:21.634507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501be98d-a298-4d1c-ad94-884be527c29c admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:21.692799 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8228f-c125-44a3-b685-869d7117c092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:21.693559 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8228f-c125-44a3-b685-869d7117c092 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:21.693879 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8228f-c125-44a3-b685-869d7117c092 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:21.694154 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8228f-c125-44a3-b685-869d7117c092 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:21.694859 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8228f-c125-44a3-b685-869d7117c092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:21.696813 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8228f-c125-44a3-b685-869d7117c092 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:21.719122 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1273/2561] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:54:21 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:54:21.723854 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac314c-3a78-4a1a-a855-6249448ee5f5 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:21.734283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac314c-3a78-4a1a-a855-6249448ee5f5 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:21.739505 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876c877-5563-485d-8666-c14ab82f1ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:21.740159 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876c877-5563-485d-8666-c14ab82f1ffd admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:21.740444 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876c877-5563-485d-8666-c14ab82f1ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:21.740720 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876c877-5563-485d-8666-c14ab82f1ffd admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:21.741447 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c876c877-5563-485d-8666-c14ab82f1ffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:21.743047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c876c877-5563-485d-8666-c14ab82f1ffd admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:21.754652 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1289/2562] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:54:21.758882 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8228f-c125-44a3-b685-869d7117c092 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:21.769111 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8228f-c125-44a3-b685-869d7117c092 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:21.773960 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a5670-ddcf-40ab-96b4-406f5bccb99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:21.774616 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a5670-ddcf-40ab-96b4-406f5bccb99c admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:21.774903 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a5670-ddcf-40ab-96b4-406f5bccb99c admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:21.775183 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a5670-ddcf-40ab-96b4-406f5bccb99c admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:21.775943 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a5670-ddcf-40ab-96b4-406f5bccb99c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:21.777532 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a5670-ddcf-40ab-96b4-406f5bccb99c admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:21.788098 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1274/2563] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:54:21.792184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876c877-5563-485d-8666-c14ab82f1ffd admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:21.802597 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876c877-5563-485d-8666-c14ab82f1ffd admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:21.807248 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991a401-8542-4b8a-a278-87160ad809ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:21.808148 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991a401-8542-4b8a-a278-87160ad809ea admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:21.808430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991a401-8542-4b8a-a278-87160ad809ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:21.808705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991a401-8542-4b8a-a278-87160ad809ea admin admin] PATH_INFO: `/v3/projects/ad606a36ccb649ae9ae8ec1a34ad9336/users/8ab7f7749bdd49ae9fa0ba9063a8e66b/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:21.823211 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991a401-8542-4b8a-a278-87160ad809ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad606a36ccb649ae9ae8ec1a34ad9336, user_id=8ab7f7749bdd49ae9fa0ba9063a8e66b, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:21.825901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991a401-8542-4b8a-a278-87160ad809ea admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:21.838566 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1290/2564] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:54:21 2022] PUT /identity/v3/projects/ad606a36ccb649ae9ae8ec1a34ad9336/users/8ab7f7749bdd49ae9fa0ba9063a8e66b/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:54:21.843707 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a713c64-3171-42eb-ae4f-c5e1c3e6c1c0 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:21.844027 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a713c64-3171-42eb-ae4f-c5e1c3e6c1c0 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:21.844293 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a713c64-3171-42eb-ae4f-c5e1c3e6c1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:21.862665 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-1a713c64-3171-42eb-ae4f-c5e1c3e6c1c0 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:54:21.863889 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-1a713c64-3171-42eb-ae4f-c5e1c3e6c1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:21.921430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1275/2565] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:54:21.931635 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991a401-8542-4b8a-a278-87160ad809ea admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:21.933340 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b991a401-8542-4b8a-a278-87160ad809ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:21.990237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991a401-8542-4b8a-a278-87160ad809ea admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:21.993909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ad68d-a307-471d-a968-cdec43f4e102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:21.994546 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ad68d-a307-471d-a968-cdec43f4e102 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:21.994828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ad68d-a307-471d-a968-cdec43f4e102 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:21.995102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ad68d-a307-471d-a968-cdec43f4e102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:22.004161 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ad68d-a307-471d-a968-cdec43f4e102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:22.005606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ad68d-a307-471d-a968-cdec43f4e102 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:22.015218 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1291/2566] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:54:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:54:22.666858 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed362d8-20e7-492b-b8b8-adb84ef799d2 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:22.667424 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed362d8-20e7-492b-b8b8-adb84ef799d2 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:22.667534 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed362d8-20e7-492b-b8b8-adb84ef799d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:22.695224 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-4ed362d8-20e7-492b-b8b8-adb84ef799d2 None None] MFA Rules not processed for user `8ab7f7749bdd49ae9fa0ba9063a8e66b`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:54:22.696745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-4ed362d8-20e7-492b-b8b8-adb84ef799d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:22.726778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:22.768190 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1276/2567] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:54:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:54:22.772824 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ad68d-a307-471d-a968-cdec43f4e102 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:22.782489 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ad68d-a307-471d-a968-cdec43f4e102 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:22.787460 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aebf44-3873-4860-9602-79c20321895c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:22.788188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aebf44-3873-4860-9602-79c20321895c admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:22.788534 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aebf44-3873-4860-9602-79c20321895c admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:22.788882 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aebf44-3873-4860-9602-79c20321895c admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:22.789633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aebf44-3873-4860-9602-79c20321895c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:22.791631 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aebf44-3873-4860-9602-79c20321895c admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:22.824409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1292/2568] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:54:22 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:54:22.825761 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed362d8-20e7-492b-b8b8-adb84ef799d2 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:22.837243 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed362d8-20e7-492b-b8b8-adb84ef799d2 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:22.895697 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88be14-6092-4dc9-b8f7-93c364d15104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:22.896465 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88be14-6092-4dc9-b8f7-93c364d15104 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:22.896749 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88be14-6092-4dc9-b8f7-93c364d15104 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:22.897024 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88be14-6092-4dc9-b8f7-93c364d15104 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:22.897707 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88be14-6092-4dc9-b8f7-93c364d15104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:22.899625 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88be14-6092-4dc9-b8f7-93c364d15104 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:22.920530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1277/2569] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:54:22 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:54:22.924641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aebf44-3873-4860-9602-79c20321895c admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:22.937110 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aebf44-3873-4860-9602-79c20321895c admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:22.942190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f9f63-1559-47e1-885c-938b6729826f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:22.942847 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f9f63-1559-47e1-885c-938b6729826f admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:22.943134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f9f63-1559-47e1-885c-938b6729826f admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:22.943448 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f9f63-1559-47e1-885c-938b6729826f admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:22.944176 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f9f63-1559-47e1-885c-938b6729826f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:22.945794 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f9f63-1559-47e1-885c-938b6729826f admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:22.957283 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1293/2570] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:54:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:54:22.961472 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88be14-6092-4dc9-b8f7-93c364d15104 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:22.972436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88be14-6092-4dc9-b8f7-93c364d15104 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:22.977206 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275a98b-be42-4d51-ba88-2f44e0fb4aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:22.978073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275a98b-be42-4d51-ba88-2f44e0fb4aa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:22.978356 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275a98b-be42-4d51-ba88-2f44e0fb4aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:22.978631 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275a98b-be42-4d51-ba88-2f44e0fb4aa4 admin admin] PATH_INFO: `/v3/projects/bfcaa96e3875422f962ebe72efc17bfb/users/e913a57dc59648c5a0d070825056919a/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:22.993408 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275a98b-be42-4d51-ba88-2f44e0fb4aa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfcaa96e3875422f962ebe72efc17bfb, user_id=e913a57dc59648c5a0d070825056919a, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:22.996086 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275a98b-be42-4d51-ba88-2f44e0fb4aa4 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:23.008741 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1278/2571] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:54:22 2022] PUT /identity/v3/projects/bfcaa96e3875422f962ebe72efc17bfb/users/e913a57dc59648c5a0d070825056919a/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:54:23.012738 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f9f63-1559-47e1-885c-938b6729826f admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:23.023481 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f9f63-1559-47e1-885c-938b6729826f admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:23.077879 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b53ce-5740-40d6-82bf-ffc4f5ebea67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:23.078672 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b53ce-5740-40d6-82bf-ffc4f5ebea67 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:23.079059 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b53ce-5740-40d6-82bf-ffc4f5ebea67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:23.079898 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b53ce-5740-40d6-82bf-ffc4f5ebea67 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:23.080594 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b53ce-5740-40d6-82bf-ffc4f5ebea67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:23.082217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b53ce-5740-40d6-82bf-ffc4f5ebea67 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:23.087222 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:23.092318 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1294/2572] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:54:23.097024 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275a98b-be42-4d51-ba88-2f44e0fb4aa4 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:23.107517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275a98b-be42-4d51-ba88-2f44e0fb4aa4 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:23.112251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab445e7-f9cb-427c-bd9c-96211658c5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:23.113149 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab445e7-f9cb-427c-bd9c-96211658c5d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:23.113417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab445e7-f9cb-427c-bd9c-96211658c5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:23.113697 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab445e7-f9cb-427c-bd9c-96211658c5d5 admin admin] PATH_INFO: `/v3/projects/bfcaa96e3875422f962ebe72efc17bfb/users/e913a57dc59648c5a0d070825056919a/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:23.115698 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab445e7-f9cb-427c-bd9c-96211658c5d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfcaa96e3875422f962ebe72efc17bfb, user_id=e913a57dc59648c5a0d070825056919a, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:23.118312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab445e7-f9cb-427c-bd9c-96211658c5d5 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:23.129527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1279/2573] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:54:23 2022] PUT /identity/v3/projects/bfcaa96e3875422f962ebe72efc17bfb/users/e913a57dc59648c5a0d070825056919a/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:54:23.721845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029ae2e-25d9-4579-8d47-ed9c01f8c83a None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:23.722234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029ae2e-25d9-4579-8d47-ed9c01f8c83a None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:23.722583 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029ae2e-25d9-4579-8d47-ed9c01f8c83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:23.750095 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-7029ae2e-25d9-4579-8d47-ed9c01f8c83a None None] MFA Rules not processed for user `e913a57dc59648c5a0d070825056919a`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:54:23.751921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-7029ae2e-25d9-4579-8d47-ed9c01f8c83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:23.832206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1295/2574] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:54:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:54:23.839629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab445e7-f9cb-427c-bd9c-96211658c5d5 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:23.841388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-eab445e7-f9cb-427c-bd9c-96211658c5d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:23.901592 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab445e7-f9cb-427c-bd9c-96211658c5d5 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:23.905422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb099194-ab02-41ea-9bb1-3c26b09e8b04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:23.906083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb099194-ab02-41ea-9bb1-3c26b09e8b04 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:23.906367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb099194-ab02-41ea-9bb1-3c26b09e8b04 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:23.906649 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb099194-ab02-41ea-9bb1-3c26b09e8b04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:23.941365 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:23.964921 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb099194-ab02-41ea-9bb1-3c26b09e8b04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:23.966348 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb099194-ab02-41ea-9bb1-3c26b09e8b04 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:23.975990 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1280/2575] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:54:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:54:25.330317 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029ae2e-25d9-4579-8d47-ed9c01f8c83a None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:25.339813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029ae2e-25d9-4579-8d47-ed9c01f8c83a None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:25.343330 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3886b1-4edc-444e-bd25-c63226f06e74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:25.344016 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3886b1-4edc-444e-bd25-c63226f06e74 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:25.344246 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3886b1-4edc-444e-bd25-c63226f06e74 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:25.344520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3886b1-4edc-444e-bd25-c63226f06e74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:25.353386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3886b1-4edc-444e-bd25-c63226f06e74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:25.354800 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3886b1-4edc-444e-bd25-c63226f06e74 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:25.364595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1296/2576] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:54:25 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:54:26.347511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56566564-e778-41d1-a7c0-eec9dfeb92c9 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:26.348042 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56566564-e778-41d1-a7c0-eec9dfeb92c9 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:26.348042 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56566564-e778-41d1-a7c0-eec9dfeb92c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:26.366655 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-56566564-e778-41d1-a7c0-eec9dfeb92c9 None None] MFA Rules not processed for user `239de565f79f474bb21699fd9adc96c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:54:26.367870 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-56566564-e778-41d1-a7c0-eec9dfeb92c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:26.421086 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1281/2577] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:54:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:54:35.359858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=40 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:36.414731 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=36 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:38.624452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3886b1-4edc-444e-bd25-c63226f06e74 service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:38.686796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3886b1-4edc-444e-bd25-c63226f06e74 service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:38.691442 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb58015-70cc-4ae8-bcff-14ce46ef9aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:38.692323 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb58015-70cc-4ae8-bcff-14ce46ef9aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:38.692608 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb58015-70cc-4ae8-bcff-14ce46ef9aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:38.692888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb58015-70cc-4ae8-bcff-14ce46ef9aa1 admin admin] PATH_INFO: `/v3/users/8ab7f7749bdd49ae9fa0ba9063a8e66b` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:38.694112 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb58015-70cc-4ae8-bcff-14ce46ef9aa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ab7f7749bdd49ae9fa0ba9063a8e66b)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:38.696083 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb58015-70cc-4ae8-bcff-14ce46ef9aa1 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:38.748885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-1bb58015-70cc-4ae8-bcff-14ce46ef9aa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ab7f7749bdd49ae9fa0ba9063a8e66b'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:38.763900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-1bb58015-70cc-4ae8-bcff-14ce46ef9aa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ab7f7749bdd49ae9fa0ba9063a8e66b'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:38.772009 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-1bb58015-70cc-4ae8-bcff-14ce46ef9aa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ab7f7749bdd49ae9fa0ba9063a8e66b'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:38.780175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-1bb58015-70cc-4ae8-bcff-14ce46ef9aa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ab7f7749bdd49ae9fa0ba9063a8e66b'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:38.780959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-1bb58015-70cc-4ae8-bcff-14ce46ef9aa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ab7f7749bdd49ae9fa0ba9063a8e66b'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:38.788926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1297/2578] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:54:38 2022] DELETE /identity/v3/users/8ab7f7749bdd49ae9fa0ba9063a8e66b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:54:38.989974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56566564-e778-41d1-a7c0-eec9dfeb92c9 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:39.001388 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56566564-e778-41d1-a7c0-eec9dfeb92c9 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:39.065254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a68df3-b9ed-4098-bc38-0ed7a305b004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:39.066228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a68df3-b9ed-4098-bc38-0ed7a305b004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:39.066519 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a68df3-b9ed-4098-bc38-0ed7a305b004 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:39.066795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a68df3-b9ed-4098-bc38-0ed7a305b004 admin admin] PATH_INFO: `/v3/projects/ad606a36ccb649ae9ae8ec1a34ad9336` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:39.068055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a68df3-b9ed-4098-bc38-0ed7a305b004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad606a36ccb649ae9ae8ec1a34ad9336)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:39.069997 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a68df3-b9ed-4098-bc38-0ed7a305b004 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:39.122153 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-12a68df3-b9ed-4098-bc38-0ed7a305b004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad606a36ccb649ae9ae8ec1a34ad9336', 'request_id': 'req-12a68df3-b9ed-4098-bc38-0ed7a305b004'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:39.134400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-12a68df3-b9ed-4098-bc38-0ed7a305b004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad606a36ccb649ae9ae8ec1a34ad9336', 'request_id': 'req-12a68df3-b9ed-4098-bc38-0ed7a305b004'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:39.142950 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-12a68df3-b9ed-4098-bc38-0ed7a305b004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad606a36ccb649ae9ae8ec1a34ad9336', 'request_id': 'req-12a68df3-b9ed-4098-bc38-0ed7a305b004'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:39.153705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-12a68df3-b9ed-4098-bc38-0ed7a305b004 admin admin] The token cache is being invalidate because project ad606a36ccb649ae9ae8ec1a34ad9336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:54:39.153962 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-12a68df3-b9ed-4098-bc38-0ed7a305b004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:39.154625 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-12a68df3-b9ed-4098-bc38-0ed7a305b004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:39.156214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1282/2579] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:54:38 2022] DELETE /identity/v3/projects/ad606a36ccb649ae9ae8ec1a34ad9336 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:54:39.160822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb58015-70cc-4ae8-bcff-14ce46ef9aa1 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:39.162221 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-1bb58015-70cc-4ae8-bcff-14ce46ef9aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:39.175249 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb58015-70cc-4ae8-bcff-14ce46ef9aa1 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:39.238270 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c4331-0979-4370-95a5-eb0f01eb0458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:39.239140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c4331-0979-4370-95a5-eb0f01eb0458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:39.239455 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c4331-0979-4370-95a5-eb0f01eb0458 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:39.239732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c4331-0979-4370-95a5-eb0f01eb0458 admin admin] PATH_INFO: `/v3/users/e913a57dc59648c5a0d070825056919a` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:39.240912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c4331-0979-4370-95a5-eb0f01eb0458 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e913a57dc59648c5a0d070825056919a)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:39.242825 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c4331-0979-4370-95a5-eb0f01eb0458 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:39.272059 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:39.299427 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-7f9c4331-0979-4370-95a5-eb0f01eb0458 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e913a57dc59648c5a0d070825056919a'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:39.316263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-7f9c4331-0979-4370-95a5-eb0f01eb0458 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e913a57dc59648c5a0d070825056919a'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:39.324373 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-7f9c4331-0979-4370-95a5-eb0f01eb0458 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e913a57dc59648c5a0d070825056919a'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:39.336162 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-7f9c4331-0979-4370-95a5-eb0f01eb0458 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e913a57dc59648c5a0d070825056919a'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:39.336900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-7f9c4331-0979-4370-95a5-eb0f01eb0458 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e913a57dc59648c5a0d070825056919a'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:39.343460 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1298/2580] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:54:39 2022] DELETE /identity/v3/users/e913a57dc59648c5a0d070825056919a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:54:39.533232 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a68df3-b9ed-4098-bc38-0ed7a305b004 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:39.534699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-12a68df3-b9ed-4098-bc38-0ed7a305b004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:39.554375 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a68df3-b9ed-4098-bc38-0ed7a305b004 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:39.623199 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb592c-d89e-4596-9d7d-574483c2d524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:39.624514 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb592c-d89e-4596-9d7d-574483c2d524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:39.624805 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb592c-d89e-4596-9d7d-574483c2d524 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:39.625087 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb592c-d89e-4596-9d7d-574483c2d524 admin admin] PATH_INFO: `/v3/projects/bfcaa96e3875422f962ebe72efc17bfb` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:39.626333 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb592c-d89e-4596-9d7d-574483c2d524 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfcaa96e3875422f962ebe72efc17bfb)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:39.628840 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb592c-d89e-4596-9d7d-574483c2d524 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:39.669759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:39.684713 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1bbb592c-d89e-4596-9d7d-574483c2d524 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfcaa96e3875422f962ebe72efc17bfb', 'request_id': 'req-1bbb592c-d89e-4596-9d7d-574483c2d524'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:39.692680 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1bbb592c-d89e-4596-9d7d-574483c2d524 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfcaa96e3875422f962ebe72efc17bfb', 'request_id': 'req-1bbb592c-d89e-4596-9d7d-574483c2d524'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:39.699008 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1bbb592c-d89e-4596-9d7d-574483c2d524 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfcaa96e3875422f962ebe72efc17bfb', 'request_id': 'req-1bbb592c-d89e-4596-9d7d-574483c2d524'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:39.709293 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1bbb592c-d89e-4596-9d7d-574483c2d524 admin admin] The token cache is being invalidate because project bfcaa96e3875422f962ebe72efc17bfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:54:39.709577 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1bbb592c-d89e-4596-9d7d-574483c2d524 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:39.710201 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1bbb592c-d89e-4596-9d7d-574483c2d524 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:54:39.711768 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1283/2581] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:54:39 2022] DELETE /identity/v3/projects/bfcaa96e3875422f962ebe72efc17bfb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:54:39.717596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1f909-8e00-4790-92b8-44c34577e52f None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:39.718073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1f909-8e00-4790-92b8-44c34577e52f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:39.718185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1f909-8e00-4790-92b8-44c34577e52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:39.738269 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-fee1f909-8e00-4790-92b8-44c34577e52f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:54:39.739520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-fee1f909-8e00-4790-92b8-44c34577e52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:39.800101 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1299/2582] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:54:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:54:39.808007 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a48f4-9d6f-459a-8447-e9f46bca82db None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:39.808321 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a48f4-9d6f-459a-8447-e9f46bca82db None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:39.808660 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a48f4-9d6f-459a-8447-e9f46bca82db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:39.827454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-552a48f4-9d6f-459a-8447-e9f46bca82db None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:54:39.828657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-552a48f4-9d6f-459a-8447-e9f46bca82db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:39.836623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1284/2583] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:54:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:54:39.840848 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1f909-8e00-4790-92b8-44c34577e52f None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:39.850481 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1f909-8e00-4790-92b8-44c34577e52f None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:39.855035 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92fc5e2-9d33-4419-afc1-e85c18f41e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:39.855707 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92fc5e2-9d33-4419-afc1-e85c18f41e93 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:39.855987 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92fc5e2-9d33-4419-afc1-e85c18f41e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:39.856261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92fc5e2-9d33-4419-afc1-e85c18f41e93 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:39.856975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92fc5e2-9d33-4419-afc1-e85c18f41e93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:54:39.857277 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92fc5e2-9d33-4419-afc1-e85c18f41e93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:39.858864 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92fc5e2-9d33-4419-afc1-e85c18f41e93 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:39.871306 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1300/2584] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:54:39.875565 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a48f4-9d6f-459a-8447-e9f46bca82db None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:39.890344 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a48f4-9d6f-459a-8447-e9f46bca82db None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:39.895095 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc3825-b320-49e8-8342-1ea5d4c98b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:39.895796 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc3825-b320-49e8-8342-1ea5d4c98b6d admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:39.896142 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc3825-b320-49e8-8342-1ea5d4c98b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:39.896374 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc3825-b320-49e8-8342-1ea5d4c98b6d admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:39.897066 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc3825-b320-49e8-8342-1ea5d4c98b6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:39.898965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc3825-b320-49e8-8342-1ea5d4c98b6d admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:39.924917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1285/2585] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:54:39 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:54:39.929200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92fc5e2-9d33-4419-afc1-e85c18f41e93 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:39.940971 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92fc5e2-9d33-4419-afc1-e85c18f41e93 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:40.003470 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c56e0-3442-4072-b7a1-c465983c93da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:40.004140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c56e0-3442-4072-b7a1-c465983c93da admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:40.004419 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c56e0-3442-4072-b7a1-c465983c93da admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:40.004700 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c56e0-3442-4072-b7a1-c465983c93da admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:40.005416 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c56e0-3442-4072-b7a1-c465983c93da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:40.007316 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c56e0-3442-4072-b7a1-c465983c93da admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:40.027979 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1301/2586] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:54:39 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:54:40.032478 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc3825-b320-49e8-8342-1ea5d4c98b6d admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:40.043494 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc3825-b320-49e8-8342-1ea5d4c98b6d admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:40.048312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fd79d-946e-4c8b-a22b-f24841682be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:40.049527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fd79d-946e-4c8b-a22b-f24841682be6 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:40.049799 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fd79d-946e-4c8b-a22b-f24841682be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:40.050202 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fd79d-946e-4c8b-a22b-f24841682be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:40.050931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7fd79d-946e-4c8b-a22b-f24841682be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:40.052583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7fd79d-946e-4c8b-a22b-f24841682be6 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:40.063324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1286/2587] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:54:40.067277 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c56e0-3442-4072-b7a1-c465983c93da admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:40.077536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c56e0-3442-4072-b7a1-c465983c93da admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:40.082290 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1d983-92e8-47f9-861c-e6bc38b373ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:40.082952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1d983-92e8-47f9-861c-e6bc38b373ee admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:40.083228 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1d983-92e8-47f9-861c-e6bc38b373ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:40.083548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1d983-92e8-47f9-861c-e6bc38b373ee admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:40.084269 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1d983-92e8-47f9-861c-e6bc38b373ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:40.085851 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1d983-92e8-47f9-861c-e6bc38b373ee admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:40.099995 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1302/2588] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:54:40.101200 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fd79d-946e-4c8b-a22b-f24841682be6 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:40.111975 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fd79d-946e-4c8b-a22b-f24841682be6 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:40.116643 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba2d22-e391-4a4e-be5c-3ff681329670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:40.117530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba2d22-e391-4a4e-be5c-3ff681329670 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:40.117812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba2d22-e391-4a4e-be5c-3ff681329670 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:40.118089 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba2d22-e391-4a4e-be5c-3ff681329670 admin admin] PATH_INFO: `/v3/projects/0747589f743b43759b7b3ba7bce7b000/users/abda206035394ede965aababac1999d4/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:40.134107 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba2d22-e391-4a4e-be5c-3ff681329670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0747589f743b43759b7b3ba7bce7b000, user_id=abda206035394ede965aababac1999d4, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:40.138226 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba2d22-e391-4a4e-be5c-3ff681329670 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:40.151126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1287/2589] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:54:40 2022] PUT /identity/v3/projects/0747589f743b43759b7b3ba7bce7b000/users/abda206035394ede965aababac1999d4/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:54:40.162030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76948d-dc8c-4429-b700-26af7d9d030f None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:40.162319 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76948d-dc8c-4429-b700-26af7d9d030f None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:40.162599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76948d-dc8c-4429-b700-26af7d9d030f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:40.169555 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:40.194253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-ef76948d-dc8c-4429-b700-26af7d9d030f None None] MFA Rules not processed for user `abda206035394ede965aababac1999d4`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:54:40.195522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ef76948d-dc8c-4429-b700-26af7d9d030f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:40.264771 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1303/2590] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:54:40 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:54:40.276001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba2d22-e391-4a4e-be5c-3ff681329670 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:40.278422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-96ba2d22-e391-4a4e-be5c-3ff681329670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:40.336443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba2d22-e391-4a4e-be5c-3ff681329670 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:40.340094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07239db2-3de0-4372-a0cb-6249c8f41de7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:40.340773 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07239db2-3de0-4372-a0cb-6249c8f41de7 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:40.341053 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07239db2-3de0-4372-a0cb-6249c8f41de7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:40.341328 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07239db2-3de0-4372-a0cb-6249c8f41de7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:40.350505 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07239db2-3de0-4372-a0cb-6249c8f41de7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:40.352745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07239db2-3de0-4372-a0cb-6249c8f41de7 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:40.363056 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1288/2591] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:54:40 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:54:43.866703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76948d-dc8c-4429-b700-26af7d9d030f None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:43.868342 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ef76948d-dc8c-4429-b700-26af7d9d030f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:43.929815 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76948d-dc8c-4429-b700-26af7d9d030f None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:43.941337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a912b0-c37c-4d12-b555-5df1b78320e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:43.941993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a912b0-c37c-4d12-b555-5df1b78320e1 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:43.942275 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a912b0-c37c-4d12-b555-5df1b78320e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:43.942550 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a912b0-c37c-4d12-b555-5df1b78320e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:43.944405 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-33a912b0-c37c-4d12-b555-5df1b78320e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:44.006704 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a912b0-c37c-4d12-b555-5df1b78320e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:44.008185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a912b0-c37c-4d12-b555-5df1b78320e1 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:44.018708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1304/2592] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:54:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:54:50.358216 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:54.013637 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=80 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:54:56.001353 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07239db2-3de0-4372-a0cb-6249c8f41de7 service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:54:56.010865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07239db2-3de0-4372-a0cb-6249c8f41de7 service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:54:56.014697 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2169f-434e-438d-accd-cd1a0fb9b391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:54:56.015332 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2169f-434e-438d-accd-cd1a0fb9b391 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:54:56.015648 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2169f-434e-438d-accd-cd1a0fb9b391 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:54:56.015924 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2169f-434e-438d-accd-cd1a0fb9b391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:54:56.018203 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-7eb2169f-434e-438d-accd-cd1a0fb9b391 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:54:56.081409 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2169f-434e-438d-accd-cd1a0fb9b391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:54:56.082842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb2169f-434e-438d-accd-cd1a0fb9b391 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:54:56.092672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1289/2593] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:54:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:55:03.148785 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a912b0-c37c-4d12-b555-5df1b78320e1 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:03.152115 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-33a912b0-c37c-4d12-b555-5df1b78320e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:03.169815 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a912b0-c37c-4d12-b555-5df1b78320e1 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:03.174543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b3857-bb86-4fb7-8d1d-dbdd0b813f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:03.175525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b3857-bb86-4fb7-8d1d-dbdd0b813f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:03.175813 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b3857-bb86-4fb7-8d1d-dbdd0b813f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:03.176144 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b3857-bb86-4fb7-8d1d-dbdd0b813f72 admin admin] PATH_INFO: `/v3/users/58976549233e4bb6991258f7da1123ae` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:03.177395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b3857-bb86-4fb7-8d1d-dbdd0b813f72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58976549233e4bb6991258f7da1123ae)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:03.179341 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b3857-bb86-4fb7-8d1d-dbdd0b813f72 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:03.232526 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-021b3857-bb86-4fb7-8d1d-dbdd0b813f72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58976549233e4bb6991258f7da1123ae'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:03.251500 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-021b3857-bb86-4fb7-8d1d-dbdd0b813f72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58976549233e4bb6991258f7da1123ae'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:03.259650 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-021b3857-bb86-4fb7-8d1d-dbdd0b813f72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58976549233e4bb6991258f7da1123ae'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:03.268181 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-021b3857-bb86-4fb7-8d1d-dbdd0b813f72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58976549233e4bb6991258f7da1123ae'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:03.268826 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-021b3857-bb86-4fb7-8d1d-dbdd0b813f72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58976549233e4bb6991258f7da1123ae'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:03.275203 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1305/2594] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:55:03 2022] DELETE /identity/v3/users/58976549233e4bb6991258f7da1123ae => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:55:03.479010 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2169f-434e-438d-accd-cd1a0fb9b391 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:03.489999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2169f-434e-438d-accd-cd1a0fb9b391 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:03.554453 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbb549-6c09-4428-8cdb-0acc42eb8cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:03.555319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbb549-6c09-4428-8cdb-0acc42eb8cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:03.555632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbb549-6c09-4428-8cdb-0acc42eb8cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:03.555906 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbb549-6c09-4428-8cdb-0acc42eb8cf8 admin admin] PATH_INFO: `/v3/projects/4845f1df49c64dcc8c4a48691fae9a27` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:03.557097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbb549-6c09-4428-8cdb-0acc42eb8cf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4845f1df49c64dcc8c4a48691fae9a27)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:03.559001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbb549-6c09-4428-8cdb-0acc42eb8cf8 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:03.610700 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-92dbb549-6c09-4428-8cdb-0acc42eb8cf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4845f1df49c64dcc8c4a48691fae9a27', 'request_id': 'req-92dbb549-6c09-4428-8cdb-0acc42eb8cf8'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:03.618817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-92dbb549-6c09-4428-8cdb-0acc42eb8cf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4845f1df49c64dcc8c4a48691fae9a27', 'request_id': 'req-92dbb549-6c09-4428-8cdb-0acc42eb8cf8'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:03.625282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-92dbb549-6c09-4428-8cdb-0acc42eb8cf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4845f1df49c64dcc8c4a48691fae9a27', 'request_id': 'req-92dbb549-6c09-4428-8cdb-0acc42eb8cf8'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:03.640041 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-92dbb549-6c09-4428-8cdb-0acc42eb8cf8 admin admin] The token cache is being invalidate because project 4845f1df49c64dcc8c4a48691fae9a27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:55:03.640361 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-92dbb549-6c09-4428-8cdb-0acc42eb8cf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:03.641021 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-92dbb549-6c09-4428-8cdb-0acc42eb8cf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:03.642598 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1290/2595] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:55:03 2022] DELETE /identity/v3/projects/4845f1df49c64dcc8c4a48691fae9a27 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:55:03.646527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b3857-bb86-4fb7-8d1d-dbdd0b813f72 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:03.647952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-021b3857-bb86-4fb7-8d1d-dbdd0b813f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:03.664986 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b3857-bb86-4fb7-8d1d-dbdd0b813f72 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:03.722330 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00b830-a642-4037-bded-7c66b86bc8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:03.723206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00b830-a642-4037-bded-7c66b86bc8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:03.723529 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00b830-a642-4037-bded-7c66b86bc8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:03.723812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea00b830-a642-4037-bded-7c66b86bc8d7 admin admin] PATH_INFO: `/v3/users/1ab42aa23f0947479254a5f79aaa226c` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:03.725010 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea00b830-a642-4037-bded-7c66b86bc8d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ab42aa23f0947479254a5f79aaa226c)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:03.727040 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea00b830-a642-4037-bded-7c66b86bc8d7 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:03.776894 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-ea00b830-a642-4037-bded-7c66b86bc8d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ab42aa23f0947479254a5f79aaa226c'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:03.790295 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-ea00b830-a642-4037-bded-7c66b86bc8d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ab42aa23f0947479254a5f79aaa226c'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:03.797658 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-ea00b830-a642-4037-bded-7c66b86bc8d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ab42aa23f0947479254a5f79aaa226c'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:03.800425 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:55:03.807794 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-ea00b830-a642-4037-bded-7c66b86bc8d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ab42aa23f0947479254a5f79aaa226c'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:03.808519 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-ea00b830-a642-4037-bded-7c66b86bc8d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ab42aa23f0947479254a5f79aaa226c'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:03.815205 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1306/2596] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:55:03 2022] DELETE /identity/v3/users/1ab42aa23f0947479254a5f79aaa226c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:55:04.000637 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbb549-6c09-4428-8cdb-0acc42eb8cf8 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:04.002413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-92dbb549-6c09-4428-8cdb-0acc42eb8cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:04.020645 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbb549-6c09-4428-8cdb-0acc42eb8cf8 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:04.047457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:55:04.092575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05272d-bdcb-4ee1-88c2-a56194de6e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:04.093449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05272d-bdcb-4ee1-88c2-a56194de6e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:04.093728 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05272d-bdcb-4ee1-88c2-a56194de6e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:04.094005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05272d-bdcb-4ee1-88c2-a56194de6e28 admin admin] PATH_INFO: `/v3/projects/97aa107a7cc34f0e8d55b4586f339fbe` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:04.095251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05272d-bdcb-4ee1-88c2-a56194de6e28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97aa107a7cc34f0e8d55b4586f339fbe)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:04.097799 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05272d-bdcb-4ee1-88c2-a56194de6e28 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:04.150304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1e05272d-bdcb-4ee1-88c2-a56194de6e28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97aa107a7cc34f0e8d55b4586f339fbe', 'request_id': 'req-1e05272d-bdcb-4ee1-88c2-a56194de6e28'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:04.157613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1e05272d-bdcb-4ee1-88c2-a56194de6e28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97aa107a7cc34f0e8d55b4586f339fbe', 'request_id': 'req-1e05272d-bdcb-4ee1-88c2-a56194de6e28'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:04.166030 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1e05272d-bdcb-4ee1-88c2-a56194de6e28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97aa107a7cc34f0e8d55b4586f339fbe', 'request_id': 'req-1e05272d-bdcb-4ee1-88c2-a56194de6e28'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:04.176671 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1e05272d-bdcb-4ee1-88c2-a56194de6e28 admin admin] The token cache is being invalidate because project 97aa107a7cc34f0e8d55b4586f339fbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:55:04.176959 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1e05272d-bdcb-4ee1-88c2-a56194de6e28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:04.177628 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1e05272d-bdcb-4ee1-88c2-a56194de6e28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:04.179687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1291/2597] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:55:03 2022] DELETE /identity/v3/projects/97aa107a7cc34f0e8d55b4586f339fbe => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:55:13.814496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=4 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:55:14.173563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:55:26.879717 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00b830-a642-4037-bded-7c66b86bc8d7 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:26.881141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ea00b830-a642-4037-bded-7c66b86bc8d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:26.895322 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea00b830-a642-4037-bded-7c66b86bc8d7 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:26.952490 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afacdf5-82ca-47b7-a869-305c0009db8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:26.953367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afacdf5-82ca-47b7-a869-305c0009db8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:26.953649 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afacdf5-82ca-47b7-a869-305c0009db8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:26.953926 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afacdf5-82ca-47b7-a869-305c0009db8b admin admin] PATH_INFO: `/v3/users/abda206035394ede965aababac1999d4` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:26.955217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afacdf5-82ca-47b7-a869-305c0009db8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=abda206035394ede965aababac1999d4)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:26.957791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afacdf5-82ca-47b7-a869-305c0009db8b admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:27.003009 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6afacdf5-82ca-47b7-a869-305c0009db8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abda206035394ede965aababac1999d4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:27.015337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6afacdf5-82ca-47b7-a869-305c0009db8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abda206035394ede965aababac1999d4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:27.022684 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6afacdf5-82ca-47b7-a869-305c0009db8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abda206035394ede965aababac1999d4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:27.029634 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6afacdf5-82ca-47b7-a869-305c0009db8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abda206035394ede965aababac1999d4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:27.030270 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-6afacdf5-82ca-47b7-a869-305c0009db8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abda206035394ede965aababac1999d4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:27.036578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1307/2598] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:55:26 2022] DELETE /identity/v3/users/abda206035394ede965aababac1999d4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:55:27.041038 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc72b8-c146-4f86-979a-49bc71144094 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:27.041549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc72b8-c146-4f86-979a-49bc71144094 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:27.041661 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc72b8-c146-4f86-979a-49bc71144094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:27.059341 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-82bc72b8-c146-4f86-979a-49bc71144094 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:55:27.060572 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-82bc72b8-c146-4f86-979a-49bc71144094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:27.118316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1292/2599] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:55:27.127934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afacdf5-82ca-47b7-a869-305c0009db8b admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:27.129420 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-6afacdf5-82ca-47b7-a869-305c0009db8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:27.186855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afacdf5-82ca-47b7-a869-305c0009db8b admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:27.190486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07b468-9a22-483e-bc4e-749bb694dcad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:27.191117 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07b468-9a22-483e-bc4e-749bb694dcad service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:27.191431 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07b468-9a22-483e-bc4e-749bb694dcad service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:27.191705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07b468-9a22-483e-bc4e-749bb694dcad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:27.200766 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07b468-9a22-483e-bc4e-749bb694dcad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:27.202169 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07b468-9a22-483e-bc4e-749bb694dcad service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:27.211604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1308/2600] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:55:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:55:27.475733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc72b8-c146-4f86-979a-49bc71144094 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:27.477395 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-82bc72b8-c146-4f86-979a-49bc71144094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:27.489897 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc72b8-c146-4f86-979a-49bc71144094 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:27.494524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbd4c4-ce78-4820-aa6e-ef9ac0992979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:27.495418 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbd4c4-ce78-4820-aa6e-ef9ac0992979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:27.495707 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbd4c4-ce78-4820-aa6e-ef9ac0992979 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:27.495990 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbd4c4-ce78-4820-aa6e-ef9ac0992979 admin admin] PATH_INFO: `/v3/projects/0747589f743b43759b7b3ba7bce7b000` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:27.497334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbd4c4-ce78-4820-aa6e-ef9ac0992979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0747589f743b43759b7b3ba7bce7b000)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:27.499245 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbd4c4-ce78-4820-aa6e-ef9ac0992979 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:27.549131 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-92dbd4c4-ce78-4820-aa6e-ef9ac0992979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0747589f743b43759b7b3ba7bce7b000', 'request_id': 'req-92dbd4c4-ce78-4820-aa6e-ef9ac0992979'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:27.556873 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-92dbd4c4-ce78-4820-aa6e-ef9ac0992979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0747589f743b43759b7b3ba7bce7b000', 'request_id': 'req-92dbd4c4-ce78-4820-aa6e-ef9ac0992979'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:27.563192 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-92dbd4c4-ce78-4820-aa6e-ef9ac0992979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0747589f743b43759b7b3ba7bce7b000', 'request_id': 'req-92dbd4c4-ce78-4820-aa6e-ef9ac0992979'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:27.573552 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-92dbd4c4-ce78-4820-aa6e-ef9ac0992979 admin admin] The token cache is being invalidate because project 0747589f743b43759b7b3ba7bce7b000 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:55:27.573839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-92dbd4c4-ce78-4820-aa6e-ef9ac0992979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:27.574467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-92dbd4c4-ce78-4820-aa6e-ef9ac0992979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:27.576086 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1293/2601] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:55:27 2022] DELETE /identity/v3/projects/0747589f743b43759b7b3ba7bce7b000 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:55:27.581004 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa4c34-e7ba-4455-b224-679c091cab7e None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:27.581293 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa4c34-e7ba-4455-b224-679c091cab7e None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:27.582110 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa4c34-e7ba-4455-b224-679c091cab7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:27.600482 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-00fa4c34-e7ba-4455-b224-679c091cab7e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:55:27.602372 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-00fa4c34-e7ba-4455-b224-679c091cab7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:27.621302 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:55:27.664680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1309/2602] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:55:27.672216 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f515b-6f0d-4716-945c-acafdf857a6d None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:27.672507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f515b-6f0d-4716-945c-acafdf857a6d None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:27.672784 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f515b-6f0d-4716-945c-acafdf857a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:27.690663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-4c2f515b-6f0d-4716-945c-acafdf857a6d None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:55:27.692404 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-4c2f515b-6f0d-4716-945c-acafdf857a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:27.700354 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1294/2603] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:55:27.704703 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa4c34-e7ba-4455-b224-679c091cab7e None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:27.715831 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa4c34-e7ba-4455-b224-679c091cab7e None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:27.720398 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67b89a-00da-48e3-a536-6eb19c3690b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:27.721049 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67b89a-00da-48e3-a536-6eb19c3690b0 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:27.721329 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67b89a-00da-48e3-a536-6eb19c3690b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:27.721604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67b89a-00da-48e3-a536-6eb19c3690b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:27.722355 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67b89a-00da-48e3-a536-6eb19c3690b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:55:27.722654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67b89a-00da-48e3-a536-6eb19c3690b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:27.724284 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67b89a-00da-48e3-a536-6eb19c3690b0 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:27.736074 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1310/2604] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:55:27.741192 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f515b-6f0d-4716-945c-acafdf857a6d None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:27.750620 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f515b-6f0d-4716-945c-acafdf857a6d None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:27.756319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bd6f1-a861-4f6f-b8ae-1f78eb2eccf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:27.757384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bd6f1-a861-4f6f-b8ae-1f78eb2eccf6 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:27.757681 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bd6f1-a861-4f6f-b8ae-1f78eb2eccf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:27.758026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bd6f1-a861-4f6f-b8ae-1f78eb2eccf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:27.758753 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bd6f1-a861-4f6f-b8ae-1f78eb2eccf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:27.761161 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bd6f1-a861-4f6f-b8ae-1f78eb2eccf6 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:27.788275 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1295/2605] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:55:27 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:55:27.792432 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67b89a-00da-48e3-a536-6eb19c3690b0 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:27.802400 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67b89a-00da-48e3-a536-6eb19c3690b0 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:27.860702 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97beff21-4b11-4421-9c19-a4dd2d056277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:27.861901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97beff21-4b11-4421-9c19-a4dd2d056277 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:27.862425 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97beff21-4b11-4421-9c19-a4dd2d056277 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:27.862949 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97beff21-4b11-4421-9c19-a4dd2d056277 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:27.864146 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97beff21-4b11-4421-9c19-a4dd2d056277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:27.866087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97beff21-4b11-4421-9c19-a4dd2d056277 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:27.886657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1311/2606] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:55:27 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:55:27.891108 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bd6f1-a861-4f6f-b8ae-1f78eb2eccf6 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:27.901983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bd6f1-a861-4f6f-b8ae-1f78eb2eccf6 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:27.907473 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d4c16-b4ab-45ba-86d3-68d4a58a51a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:27.908134 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d4c16-b4ab-45ba-86d3-68d4a58a51a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:27.908421 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d4c16-b4ab-45ba-86d3-68d4a58a51a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:27.908722 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d4c16-b4ab-45ba-86d3-68d4a58a51a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:27.909453 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d4c16-b4ab-45ba-86d3-68d4a58a51a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:27.911061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d4c16-b4ab-45ba-86d3-68d4a58a51a2 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:27.921440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1296/2607] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:55:27.925415 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97beff21-4b11-4421-9c19-a4dd2d056277 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:27.935582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97beff21-4b11-4421-9c19-a4dd2d056277 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:27.940443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52775334-81af-4b21-af1c-d481b5cd0919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:27.941597 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52775334-81af-4b21-af1c-d481b5cd0919 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:27.941886 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52775334-81af-4b21-af1c-d481b5cd0919 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:27.942294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52775334-81af-4b21-af1c-d481b5cd0919 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:27.943040 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52775334-81af-4b21-af1c-d481b5cd0919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:27.945142 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52775334-81af-4b21-af1c-d481b5cd0919 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:27.954809 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1312/2608] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:55:27.959603 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d4c16-b4ab-45ba-86d3-68d4a58a51a2 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:27.969501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d4c16-b4ab-45ba-86d3-68d4a58a51a2 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:27.974280 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d1c5f-c7b8-42d6-826e-b9f81900e2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:27.975150 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d1c5f-c7b8-42d6-826e-b9f81900e2f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:27.975464 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d1c5f-c7b8-42d6-826e-b9f81900e2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:27.975802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d1c5f-c7b8-42d6-826e-b9f81900e2f2 admin admin] PATH_INFO: `/v3/projects/92f54ab236324100ae1b8b9660f403d6/users/5dde3e5a2826404687604580f3a36494/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:27.990961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d1c5f-c7b8-42d6-826e-b9f81900e2f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f54ab236324100ae1b8b9660f403d6, user_id=5dde3e5a2826404687604580f3a36494, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:27.993712 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d1c5f-c7b8-42d6-826e-b9f81900e2f2 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:28.007484 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1297/2609] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:55:27 2022] PUT /identity/v3/projects/92f54ab236324100ae1b8b9660f403d6/users/5dde3e5a2826404687604580f3a36494/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:55:28.017854 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70b68f-f042-4520-ba5b-ebd6e437406e None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:28.018144 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70b68f-f042-4520-ba5b-ebd6e437406e None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:28.018422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70b68f-f042-4520-ba5b-ebd6e437406e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:28.046732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-9e70b68f-f042-4520-ba5b-ebd6e437406e None None] MFA Rules not processed for user `5dde3e5a2826404687604580f3a36494`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:55:28.047974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-9e70b68f-f042-4520-ba5b-ebd6e437406e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:28.111474 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1313/2610] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:55:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:55:28.123002 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d1c5f-c7b8-42d6-826e-b9f81900e2f2 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:28.124770 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-313d1c5f-c7b8-42d6-826e-b9f81900e2f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:28.147008 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:55:28.186879 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d1c5f-c7b8-42d6-826e-b9f81900e2f2 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:28.190538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172aee6-5eaa-4180-ae19-6074b381a8f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:28.191856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172aee6-5eaa-4180-ae19-6074b381a8f2 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:28.192138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172aee6-5eaa-4180-ae19-6074b381a8f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:28.192419 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172aee6-5eaa-4180-ae19-6074b381a8f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:28.201176 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172aee6-5eaa-4180-ae19-6074b381a8f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:28.202614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172aee6-5eaa-4180-ae19-6074b381a8f2 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:28.211491 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1298/2611] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:55:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:55:34.851514 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70b68f-f042-4520-ba5b-ebd6e437406e None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:34.852950 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-9e70b68f-f042-4520-ba5b-ebd6e437406e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:34.865328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70b68f-f042-4520-ba5b-ebd6e437406e None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:34.868954 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1a0f7-969f-423c-8015-e5ff540f42fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:34.869593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1a0f7-969f-423c-8015-e5ff540f42fa service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:34.869876 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1a0f7-969f-423c-8015-e5ff540f42fa service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:34.870157 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1a0f7-969f-423c-8015-e5ff540f42fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:34.872070 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-41b1a0f7-969f-423c-8015-e5ff540f42fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:34.887547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:55:34.954696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1a0f7-969f-423c-8015-e5ff540f42fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:34.956172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1a0f7-969f-423c-8015-e5ff540f42fa service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:34.969421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1314/2612] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:55:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:55:38.207698 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=17 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:55:44.963821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=26 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:55:48.733716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172aee6-5eaa-4180-ae19-6074b381a8f2 service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:48.735270 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a172aee6-5eaa-4180-ae19-6074b381a8f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:48.793241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172aee6-5eaa-4180-ae19-6074b381a8f2 service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:48.796837 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facdf7dd-0679-49c7-af53-eca0e46c4bf6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ed133721ed044708ee362c93f564d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:48.797455 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdf7dd-0679-49c7-af53-eca0e46c4bf6 service glance] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:48.797737 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdf7dd-0679-49c7-af53-eca0e46c4bf6 service glance] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:48.798009 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facdf7dd-0679-49c7-af53-eca0e46c4bf6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:48.799825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-facdf7dd-0679-49c7-af53-eca0e46c4bf6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:48.813196 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdf7dd-0679-49c7-af53-eca0e46c4bf6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:48.814544 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facdf7dd-0679-49c7-af53-eca0e46c4bf6 service glance] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:48.826351 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1299/2613] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:55:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:55:51.602222 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1a0f7-969f-423c-8015-e5ff540f42fa service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:51.664575 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1a0f7-969f-423c-8015-e5ff540f42fa service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:51.669171 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970088d0-56b2-43ca-b94b-60143f60e7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:51.670035 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970088d0-56b2-43ca-b94b-60143f60e7dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:51.670320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970088d0-56b2-43ca-b94b-60143f60e7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:51.670596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970088d0-56b2-43ca-b94b-60143f60e7dd admin admin] PATH_INFO: `/v3/users/5dde3e5a2826404687604580f3a36494` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:51.671814 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970088d0-56b2-43ca-b94b-60143f60e7dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dde3e5a2826404687604580f3a36494)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:51.673744 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970088d0-56b2-43ca-b94b-60143f60e7dd admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:51.722864 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-970088d0-56b2-43ca-b94b-60143f60e7dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dde3e5a2826404687604580f3a36494'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:51.738358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-970088d0-56b2-43ca-b94b-60143f60e7dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dde3e5a2826404687604580f3a36494'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:51.747177 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-970088d0-56b2-43ca-b94b-60143f60e7dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dde3e5a2826404687604580f3a36494'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:51.755050 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-970088d0-56b2-43ca-b94b-60143f60e7dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dde3e5a2826404687604580f3a36494'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:51.755742 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-970088d0-56b2-43ca-b94b-60143f60e7dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dde3e5a2826404687604580f3a36494'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:51.763847 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1315/2614] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:55:51 2022] DELETE /identity/v3/users/5dde3e5a2826404687604580f3a36494 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:55:51.768451 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1441fb-1aff-4de2-ad28-e60b7bfdf199 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:51.768947 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1441fb-1aff-4de2-ad28-e60b7bfdf199 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:51.769057 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1441fb-1aff-4de2-ad28-e60b7bfdf199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:51.786382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-5b1441fb-1aff-4de2-ad28-e60b7bfdf199 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:55:51.787746 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-5b1441fb-1aff-4de2-ad28-e60b7bfdf199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:51.846848 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1300/2615] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:55:51.857531 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970088d0-56b2-43ca-b94b-60143f60e7dd admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:51.859238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-970088d0-56b2-43ca-b94b-60143f60e7dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:51.919250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970088d0-56b2-43ca-b94b-60143f60e7dd admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:51.922944 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b32b5-9532-4eb9-8fb5-cd3f49b76177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:51.923625 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b32b5-9532-4eb9-8fb5-cd3f49b76177 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:51.923911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b32b5-9532-4eb9-8fb5-cd3f49b76177 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:51.924190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b32b5-9532-4eb9-8fb5-cd3f49b76177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:51.934312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b32b5-9532-4eb9-8fb5-cd3f49b76177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:51.935791 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b32b5-9532-4eb9-8fb5-cd3f49b76177 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:51.945331 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1316/2616] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:55:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:55:52.211654 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1441fb-1aff-4de2-ad28-e60b7bfdf199 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:52.213255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-5b1441fb-1aff-4de2-ad28-e60b7bfdf199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:52.226354 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1441fb-1aff-4de2-ad28-e60b7bfdf199 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:52.231632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bdd04-7a88-406f-8e9e-1890f44c7bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:52.232504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bdd04-7a88-406f-8e9e-1890f44c7bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:52.232790 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bdd04-7a88-406f-8e9e-1890f44c7bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:52.233066 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bdd04-7a88-406f-8e9e-1890f44c7bcb admin admin] PATH_INFO: `/v3/projects/92f54ab236324100ae1b8b9660f403d6` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:52.234422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908bdd04-7a88-406f-8e9e-1890f44c7bcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=92f54ab236324100ae1b8b9660f403d6)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:52.236380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908bdd04-7a88-406f-8e9e-1890f44c7bcb admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:52.284204 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-908bdd04-7a88-406f-8e9e-1890f44c7bcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92f54ab236324100ae1b8b9660f403d6', 'request_id': 'req-908bdd04-7a88-406f-8e9e-1890f44c7bcb'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:52.291751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-908bdd04-7a88-406f-8e9e-1890f44c7bcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92f54ab236324100ae1b8b9660f403d6', 'request_id': 'req-908bdd04-7a88-406f-8e9e-1890f44c7bcb'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:52.298210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-908bdd04-7a88-406f-8e9e-1890f44c7bcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92f54ab236324100ae1b8b9660f403d6', 'request_id': 'req-908bdd04-7a88-406f-8e9e-1890f44c7bcb'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:52.309299 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-908bdd04-7a88-406f-8e9e-1890f44c7bcb admin admin] The token cache is being invalidate because project 92f54ab236324100ae1b8b9660f403d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:55:52.309592 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-908bdd04-7a88-406f-8e9e-1890f44c7bcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:52.310269 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-908bdd04-7a88-406f-8e9e-1890f44c7bcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:55:52.311911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1301/2617] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:55:52 2022] DELETE /identity/v3/projects/92f54ab236324100ae1b8b9660f403d6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:55:52.317012 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db5324-b1cc-4350-9fb4-6bbe5767ca44 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:52.317303 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db5324-b1cc-4350-9fb4-6bbe5767ca44 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:52.318809 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db5324-b1cc-4350-9fb4-6bbe5767ca44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:52.338271 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-88db5324-b1cc-4350-9fb4-6bbe5767ca44 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:55:52.339525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-88db5324-b1cc-4350-9fb4-6bbe5767ca44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:52.356091 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:55:52.400355 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1317/2618] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:55:52.464772 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccde8d-664d-49da-ba19-40e5dc03b0b6 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:52.465057 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccde8d-664d-49da-ba19-40e5dc03b0b6 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:52.465335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccde8d-664d-49da-ba19-40e5dc03b0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:52.483620 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-0dccde8d-664d-49da-ba19-40e5dc03b0b6 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:55:52.484816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-0dccde8d-664d-49da-ba19-40e5dc03b0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:52.492650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1302/2619] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:55:52.497043 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db5324-b1cc-4350-9fb4-6bbe5767ca44 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:52.510876 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db5324-b1cc-4350-9fb4-6bbe5767ca44 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:52.515919 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec66fd-7d6d-474b-aa13-032b758f418c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:52.516582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec66fd-7d6d-474b-aa13-032b758f418c admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:52.516865 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec66fd-7d6d-474b-aa13-032b758f418c admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:52.517143 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec66fd-7d6d-474b-aa13-032b758f418c admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:52.517869 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec66fd-7d6d-474b-aa13-032b758f418c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:55:52.518203 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec66fd-7d6d-474b-aa13-032b758f418c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:52.519838 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aec66fd-7d6d-474b-aa13-032b758f418c admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:52.533409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1318/2620] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:55:52.538132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccde8d-664d-49da-ba19-40e5dc03b0b6 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:52.548357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccde8d-664d-49da-ba19-40e5dc03b0b6 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:52.553088 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a673f9-6bbc-4bbe-a4e5-478c7cf0521a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:52.553871 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a673f9-6bbc-4bbe-a4e5-478c7cf0521a admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:52.554157 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a673f9-6bbc-4bbe-a4e5-478c7cf0521a admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:52.554434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a673f9-6bbc-4bbe-a4e5-478c7cf0521a admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:52.555893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a673f9-6bbc-4bbe-a4e5-478c7cf0521a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:52.557796 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a673f9-6bbc-4bbe-a4e5-478c7cf0521a admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:52.578022 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:55:52.586673 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1303/2621] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:55:52 2022] POST /identity/v3/projects => generated 399 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:55:52.590803 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec66fd-7d6d-474b-aa13-032b758f418c admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:52.602193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec66fd-7d6d-474b-aa13-032b758f418c admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:52.661788 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379530e-71d9-4ee7-918b-bde914d67c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:52.662448 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379530e-71d9-4ee7-918b-bde914d67c7d admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:52.662725 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379530e-71d9-4ee7-918b-bde914d67c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:52.663003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379530e-71d9-4ee7-918b-bde914d67c7d admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:52.664755 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5379530e-71d9-4ee7-918b-bde914d67c7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:52.666318 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5379530e-71d9-4ee7-918b-bde914d67c7d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:52.689591 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1319/2622] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:55:52 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:55:52.691071 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a673f9-6bbc-4bbe-a4e5-478c7cf0521a admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:52.702450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a673f9-6bbc-4bbe-a4e5-478c7cf0521a admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:52.707362 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d82349-c0c8-4d75-a617-3a7a061c679e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:52.708165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d82349-c0c8-4d75-a617-3a7a061c679e admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:52.708458 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d82349-c0c8-4d75-a617-3a7a061c679e admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:52.708740 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d82349-c0c8-4d75-a617-3a7a061c679e admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:52.709468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d82349-c0c8-4d75-a617-3a7a061c679e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:52.711592 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d82349-c0c8-4d75-a617-3a7a061c679e admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:52.722096 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1304/2623] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:55:52.726054 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379530e-71d9-4ee7-918b-bde914d67c7d admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:52.736601 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379530e-71d9-4ee7-918b-bde914d67c7d admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:52.741253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6851f0-7740-402a-a3c2-72cd14baf7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:52.741901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6851f0-7740-402a-a3c2-72cd14baf7f1 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:52.742211 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6851f0-7740-402a-a3c2-72cd14baf7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:52.742490 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6851f0-7740-402a-a3c2-72cd14baf7f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:52.743216 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6851f0-7740-402a-a3c2-72cd14baf7f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:52.744849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6851f0-7740-402a-a3c2-72cd14baf7f1 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:52.754687 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1320/2624] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:55:52.758736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d82349-c0c8-4d75-a617-3a7a061c679e admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:52.770171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d82349-c0c8-4d75-a617-3a7a061c679e admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:52.775471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac0874-5b7a-41ed-b2de-4485fd7b7e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:52.776556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac0874-5b7a-41ed-b2de-4485fd7b7e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:52.776842 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac0874-5b7a-41ed-b2de-4485fd7b7e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:52.777116 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac0874-5b7a-41ed-b2de-4485fd7b7e2e admin admin] PATH_INFO: `/v3/projects/59375ed1b1a647a8be18ea6ddc6b557b/users/a01ad192f664414c87afc7a4b2e13db6/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:52.792860 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac0874-5b7a-41ed-b2de-4485fd7b7e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=59375ed1b1a647a8be18ea6ddc6b557b, user_id=a01ad192f664414c87afc7a4b2e13db6, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:52.795570 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac0874-5b7a-41ed-b2de-4485fd7b7e2e admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:52.808894 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1305/2625] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:55:52 2022] PUT /identity/v3/projects/59375ed1b1a647a8be18ea6ddc6b557b/users/a01ad192f664414c87afc7a4b2e13db6/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:55:52.818890 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17693649-12fd-4b72-88bb-824fc4fd6914 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:52.819179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17693649-12fd-4b72-88bb-824fc4fd6914 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:52.819494 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17693649-12fd-4b72-88bb-824fc4fd6914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:52.847692 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-17693649-12fd-4b72-88bb-824fc4fd6914 None None] MFA Rules not processed for user `a01ad192f664414c87afc7a4b2e13db6`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:55:52.848944 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-17693649-12fd-4b72-88bb-824fc4fd6914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:52.916639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1321/2626] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:55:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:55:52.928606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac0874-5b7a-41ed-b2de-4485fd7b7e2e admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:55:52.930562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c7ac0874-5b7a-41ed-b2de-4485fd7b7e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:55:52.988023 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac0874-5b7a-41ed-b2de-4485fd7b7e2e admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:55:52.991811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1e0a4-67c1-4111-8fd1-56a5dbdc79a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:55:52.992491 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1e0a4-67c1-4111-8fd1-56a5dbdc79a7 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:55:52.992802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1e0a4-67c1-4111-8fd1-56a5dbdc79a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:55:52.993083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1e0a4-67c1-4111-8fd1-56a5dbdc79a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:55:53.001305 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1e0a4-67c1-4111-8fd1-56a5dbdc79a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:55:53.002730 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1e0a4-67c1-4111-8fd1-56a5dbdc79a7 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:55:53.011964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1306/2627] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:55:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:56:02.090032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17693649-12fd-4b72-88bb-824fc4fd6914 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:02.108294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:02.154208 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17693649-12fd-4b72-88bb-824fc4fd6914 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:02.159006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b17150-d443-46c5-b939-06e2f0047eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:02.159901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b17150-d443-46c5-b939-06e2f0047eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:02.160181 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b17150-d443-46c5-b939-06e2f0047eec admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:02.160454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b17150-d443-46c5-b939-06e2f0047eec admin admin] PATH_INFO: `/v3/users/a01ad192f664414c87afc7a4b2e13db6` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:02.161663 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b17150-d443-46c5-b939-06e2f0047eec admin admin] RBAC: Authorizing `identity:delete_user(user_id=a01ad192f664414c87afc7a4b2e13db6)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:02.163636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b17150-d443-46c5-b939-06e2f0047eec admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:02.216298 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d0b17150-d443-46c5-b939-06e2f0047eec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a01ad192f664414c87afc7a4b2e13db6'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:02.227978 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d0b17150-d443-46c5-b939-06e2f0047eec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a01ad192f664414c87afc7a4b2e13db6'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:02.235162 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d0b17150-d443-46c5-b939-06e2f0047eec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a01ad192f664414c87afc7a4b2e13db6'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:02.241882 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d0b17150-d443-46c5-b939-06e2f0047eec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a01ad192f664414c87afc7a4b2e13db6'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:02.242522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-d0b17150-d443-46c5-b939-06e2f0047eec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a01ad192f664414c87afc7a4b2e13db6'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:02.248300 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1322/2628] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:56:02 2022] DELETE /identity/v3/users/a01ad192f664414c87afc7a4b2e13db6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:02.252748 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234d05c-07fb-41e0-a4ac-7b9687736df6 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:02.253235 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234d05c-07fb-41e0-a4ac-7b9687736df6 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:02.253351 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2234d05c-07fb-41e0-a4ac-7b9687736df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:02.271126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-2234d05c-07fb-41e0-a4ac-7b9687736df6 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:02.272367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2234d05c-07fb-41e0-a4ac-7b9687736df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:02.330396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1307/2629] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:02.339964 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b17150-d443-46c5-b939-06e2f0047eec admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:02.341339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d0b17150-d443-46c5-b939-06e2f0047eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:02.397343 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b17150-d443-46c5-b939-06e2f0047eec admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:02.401520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346c781-e430-475d-8048-0cb311f80016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:02.402162 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346c781-e430-475d-8048-0cb311f80016 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:02.402518 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346c781-e430-475d-8048-0cb311f80016 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:02.402698 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346c781-e430-475d-8048-0cb311f80016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:02.411858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346c781-e430-475d-8048-0cb311f80016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:02.413290 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346c781-e430-475d-8048-0cb311f80016 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:02.423309 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1323/2630] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:56:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:56:02.683080 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234d05c-07fb-41e0-a4ac-7b9687736df6 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:02.684515 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2234d05c-07fb-41e0-a4ac-7b9687736df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:02.699398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2234d05c-07fb-41e0-a4ac-7b9687736df6 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:02.704312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e2ffc-6ef5-4974-8ead-4f6b2485bd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:02.706105 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e2ffc-6ef5-4974-8ead-4f6b2485bd43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:02.706391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e2ffc-6ef5-4974-8ead-4f6b2485bd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:02.706672 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e2ffc-6ef5-4974-8ead-4f6b2485bd43 admin admin] PATH_INFO: `/v3/projects/59375ed1b1a647a8be18ea6ddc6b557b` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:02.707951 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0e2ffc-6ef5-4974-8ead-4f6b2485bd43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59375ed1b1a647a8be18ea6ddc6b557b)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:02.709855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0e2ffc-6ef5-4974-8ead-4f6b2485bd43 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:02.756203 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-bd0e2ffc-6ef5-4974-8ead-4f6b2485bd43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59375ed1b1a647a8be18ea6ddc6b557b', 'request_id': 'req-bd0e2ffc-6ef5-4974-8ead-4f6b2485bd43'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:02.764426 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-bd0e2ffc-6ef5-4974-8ead-4f6b2485bd43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59375ed1b1a647a8be18ea6ddc6b557b', 'request_id': 'req-bd0e2ffc-6ef5-4974-8ead-4f6b2485bd43'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:02.770717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-bd0e2ffc-6ef5-4974-8ead-4f6b2485bd43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59375ed1b1a647a8be18ea6ddc6b557b', 'request_id': 'req-bd0e2ffc-6ef5-4974-8ead-4f6b2485bd43'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:02.772051 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:02.784142 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-bd0e2ffc-6ef5-4974-8ead-4f6b2485bd43 admin admin] The token cache is being invalidate because project 59375ed1b1a647a8be18ea6ddc6b557b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:56:02.784432 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-bd0e2ffc-6ef5-4974-8ead-4f6b2485bd43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:02.785084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-bd0e2ffc-6ef5-4974-8ead-4f6b2485bd43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:02.786634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1308/2631] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:56:02 2022] DELETE /identity/v3/projects/59375ed1b1a647a8be18ea6ddc6b557b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:08.255807 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346c781-e430-475d-8048-0cb311f80016 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:08.257499 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d346c781-e430-475d-8048-0cb311f80016 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:08.314275 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:08.354789 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346c781-e430-475d-8048-0cb311f80016 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:08.429072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6883a-3632-4157-8f41-f60de98f94b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:08.429893 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6883a-3632-4157-8f41-f60de98f94b8 service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:08.430244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6883a-3632-4157-8f41-f60de98f94b8 service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:08.431430 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6883a-3632-4157-8f41-f60de98f94b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:08.437914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-0dc6883a-3632-4157-8f41-f60de98f94b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:08.529562 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6883a-3632-4157-8f41-f60de98f94b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:08.531676 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6883a-3632-4157-8f41-f60de98f94b8 service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:08.551465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1324/2632] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:56:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:56:12.038391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e2ffc-6ef5-4974-8ead-4f6b2485bd43 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:12.040099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-bd0e2ffc-6ef5-4974-8ead-4f6b2485bd43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:12.056638 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e2ffc-6ef5-4974-8ead-4f6b2485bd43 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:12.063963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482391d-26e8-484d-ae73-e23c66a3df99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:12.064714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482391d-26e8-484d-ae73-e23c66a3df99 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:12.065321 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482391d-26e8-484d-ae73-e23c66a3df99 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:12.065602 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482391d-26e8-484d-ae73-e23c66a3df99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:12.067654 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2482391d-26e8-484d-ae73-e23c66a3df99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:12.081832 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482391d-26e8-484d-ae73-e23c66a3df99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:12.083459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482391d-26e8-484d-ae73-e23c66a3df99 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:12.096306 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1309/2633] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:56:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:56:14.123363 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6883a-3632-4157-8f41-f60de98f94b8 service nova] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:14.124790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-0dc6883a-3632-4157-8f41-f60de98f94b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:14.175468 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=100 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:14.181265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6883a-3632-4157-8f41-f60de98f94b8 service nova] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:14.185450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787534cb-e249-4671-af25-8f24c8725ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:14.186091 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787534cb-e249-4671-af25-8f24c8725ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:14.186375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787534cb-e249-4671-af25-8f24c8725ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:14.186655 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787534cb-e249-4671-af25-8f24c8725ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:14.188434 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-787534cb-e249-4671-af25-8f24c8725ef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:14.250118 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787534cb-e249-4671-af25-8f24c8725ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:14.251525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787534cb-e249-4671-af25-8f24c8725ef4 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:14.261227 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1325/2634] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:56:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:56:16.236051 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2cdd8-eb24-4b2c-bfd4-2ca2ff304349 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:16.236518 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2cdd8-eb24-4b2c-bfd4-2ca2ff304349 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:16.236611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2cdd8-eb24-4b2c-bfd4-2ca2ff304349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:16.260065 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-79e2cdd8-eb24-4b2c-bfd4-2ca2ff304349 None None] MFA Rules not processed for user `955c44ffe1bd426e900c7405df44d1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:16.261300 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-79e2cdd8-eb24-4b2c-bfd4-2ca2ff304349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:16.322810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1310/2635] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jun 22 11:56:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:16.370376 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787534cb-e249-4671-af25-8f24c8725ef4 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:16.371984 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-787534cb-e249-4671-af25-8f24c8725ef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:16.460815 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787534cb-e249-4671-af25-8f24c8725ef4 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:16.467516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6074eac-9472-49f6-ba6a-cf0058e968c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd642e290e44e67b56eb3b8361d7af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:16.468179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6074eac-9472-49f6-ba6a-cf0058e968c6 service swift] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:16.468490 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6074eac-9472-49f6-ba6a-cf0058e968c6 service swift] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:16.468763 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6074eac-9472-49f6-ba6a-cf0058e968c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:16.477765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6074eac-9472-49f6-ba6a-cf0058e968c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:16.479453 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6074eac-9472-49f6-ba6a-cf0058e968c6 service swift] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:16.491147 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1326/2636] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jun 22 11:56:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 22 11:56:17.506187 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2cdd8-eb24-4b2c-bfd4-2ca2ff304349 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:17.506187 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-79e2cdd8-eb24-4b2c-bfd4-2ca2ff304349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:17.542593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2cdd8-eb24-4b2c-bfd4-2ca2ff304349 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:17.551046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afeab84-b8ab-4d00-ac0a-4b896806b8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:17.552832 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afeab84-b8ab-4d00-ac0a-4b896806b8b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:17.553116 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afeab84-b8ab-4d00-ac0a-4b896806b8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:17.553598 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afeab84-b8ab-4d00-ac0a-4b896806b8b1 admin admin] PATH_INFO: `/v3/users/4d7c1adbb2ad456eaf5793a15d6fcb4b` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:17.554933 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afeab84-b8ab-4d00-ac0a-4b896806b8b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d7c1adbb2ad456eaf5793a15d6fcb4b)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:17.557127 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afeab84-b8ab-4d00-ac0a-4b896806b8b1 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:17.622001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1afeab84-b8ab-4d00-ac0a-4b896806b8b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d7c1adbb2ad456eaf5793a15d6fcb4b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:17.640721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1afeab84-b8ab-4d00-ac0a-4b896806b8b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d7c1adbb2ad456eaf5793a15d6fcb4b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:17.662066 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1afeab84-b8ab-4d00-ac0a-4b896806b8b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d7c1adbb2ad456eaf5793a15d6fcb4b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:17.662705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1afeab84-b8ab-4d00-ac0a-4b896806b8b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d7c1adbb2ad456eaf5793a15d6fcb4b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:17.673339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1afeab84-b8ab-4d00-ac0a-4b896806b8b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d7c1adbb2ad456eaf5793a15d6fcb4b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:17.680336 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1311/2637] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:56:17 2022] DELETE /identity/v3/users/4d7c1adbb2ad456eaf5793a15d6fcb4b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:17.930824 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6074eac-9472-49f6-ba6a-cf0058e968c6 service swift] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:17.948918 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6074eac-9472-49f6-ba6a-cf0058e968c6 service swift] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:18.026536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61a1d7-ea52-4f09-9b8d-becc4cd0d331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:18.027452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61a1d7-ea52-4f09-9b8d-becc4cd0d331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:18.027734 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61a1d7-ea52-4f09-9b8d-becc4cd0d331 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:18.028006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61a1d7-ea52-4f09-9b8d-becc4cd0d331 admin admin] PATH_INFO: `/v3/projects/878b71f291ec4107a09e3109b6acc40b` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:18.029197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61a1d7-ea52-4f09-9b8d-becc4cd0d331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=878b71f291ec4107a09e3109b6acc40b)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:18.031243 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61a1d7-ea52-4f09-9b8d-becc4cd0d331 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:18.081502 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:18.101969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-df61a1d7-ea52-4f09-9b8d-becc4cd0d331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '878b71f291ec4107a09e3109b6acc40b', 'request_id': 'req-df61a1d7-ea52-4f09-9b8d-becc4cd0d331'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:18.109812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-df61a1d7-ea52-4f09-9b8d-becc4cd0d331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '878b71f291ec4107a09e3109b6acc40b', 'request_id': 'req-df61a1d7-ea52-4f09-9b8d-becc4cd0d331'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:18.118572 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-df61a1d7-ea52-4f09-9b8d-becc4cd0d331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '878b71f291ec4107a09e3109b6acc40b', 'request_id': 'req-df61a1d7-ea52-4f09-9b8d-becc4cd0d331'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:18.131299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-df61a1d7-ea52-4f09-9b8d-becc4cd0d331 admin admin] The token cache is being invalidate because project 878b71f291ec4107a09e3109b6acc40b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:56:18.131643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-df61a1d7-ea52-4f09-9b8d-becc4cd0d331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:18.132257 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-df61a1d7-ea52-4f09-9b8d-becc4cd0d331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:18.134082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1327/2638] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:56:17 2022] DELETE /identity/v3/projects/878b71f291ec4107a09e3109b6acc40b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:18.141205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803b710-0824-4823-9541-bb35534275de None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:18.141612 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803b710-0824-4823-9541-bb35534275de None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:18.141988 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803b710-0824-4823-9541-bb35534275de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:18.171626 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c803b710-0824-4823-9541-bb35534275de None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:18.173160 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c803b710-0824-4823-9541-bb35534275de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:18.266468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:18.284380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1312/2639] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:18.300594 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd24cd7-2256-4f20-b2a4-89fdc0c08b25 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:18.301329 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd24cd7-2256-4f20-b2a4-89fdc0c08b25 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:18.301993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd24cd7-2256-4f20-b2a4-89fdc0c08b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:18.327405 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-bbd24cd7-2256-4f20-b2a4-89fdc0c08b25 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:18.328909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-bbd24cd7-2256-4f20-b2a4-89fdc0c08b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:18.347239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1328/2640] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:18.352933 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803b710-0824-4823-9541-bb35534275de None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:18.368927 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803b710-0824-4823-9541-bb35534275de None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:18.375242 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea562726-1903-4c82-bb35-28e199d6f6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:18.376826 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea562726-1903-4c82-bb35-28e199d6f6d6 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:18.377452 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea562726-1903-4c82-bb35-28e199d6f6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:18.377961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea562726-1903-4c82-bb35-28e199d6f6d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:18.378937 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea562726-1903-4c82-bb35-28e199d6f6d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:56:18.379562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea562726-1903-4c82-bb35-28e199d6f6d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:18.381441 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea562726-1903-4c82-bb35-28e199d6f6d6 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:18.397484 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1313/2641] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:56:18.401812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd24cd7-2256-4f20-b2a4-89fdc0c08b25 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:18.413190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd24cd7-2256-4f20-b2a4-89fdc0c08b25 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:18.420277 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084917c-ea11-4ab9-923e-d4bcecdcf952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:18.421131 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084917c-ea11-4ab9-923e-d4bcecdcf952 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:18.421423 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084917c-ea11-4ab9-923e-d4bcecdcf952 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:18.421794 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084917c-ea11-4ab9-923e-d4bcecdcf952 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:18.422520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084917c-ea11-4ab9-923e-d4bcecdcf952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:18.424442 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084917c-ea11-4ab9-923e-d4bcecdcf952 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:18.470028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1329/2642] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:56:18 2022] POST /identity/v3/projects => generated 365 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:56:18.474011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea562726-1903-4c82-bb35-28e199d6f6d6 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:18.487632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea562726-1903-4c82-bb35-28e199d6f6d6 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:18.563711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1143f-1a49-459a-a508-7fbf952b0277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:18.565442 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1143f-1a49-459a-a508-7fbf952b0277 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:18.565950 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1143f-1a49-459a-a508-7fbf952b0277 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:18.566415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1143f-1a49-459a-a508-7fbf952b0277 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:18.567303 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1143f-1a49-459a-a508-7fbf952b0277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:18.569889 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1143f-1a49-459a-a508-7fbf952b0277 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:18.596182 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1314/2643] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:56:18 2022] POST /identity/v3/users => generated 283 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:56:18.601172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084917c-ea11-4ab9-923e-d4bcecdcf952 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:18.621215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084917c-ea11-4ab9-923e-d4bcecdcf952 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:18.630875 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757728df-82de-4fc5-991b-a6374081bc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:18.633066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757728df-82de-4fc5-991b-a6374081bc50 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:18.637104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757728df-82de-4fc5-991b-a6374081bc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:18.637695 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757728df-82de-4fc5-991b-a6374081bc50 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:18.638521 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757728df-82de-4fc5-991b-a6374081bc50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:18.641028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757728df-82de-4fc5-991b-a6374081bc50 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:18.674150 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1330/2644] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:56:18.675677 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1143f-1a49-459a-a508-7fbf952b0277 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:18.690817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1143f-1a49-459a-a508-7fbf952b0277 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:18.698820 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15459702-9d7f-41db-b70c-a3ffc0123ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:18.698820 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15459702-9d7f-41db-b70c-a3ffc0123ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:18.698820 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15459702-9d7f-41db-b70c-a3ffc0123ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:18.703775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15459702-9d7f-41db-b70c-a3ffc0123ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:18.704798 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15459702-9d7f-41db-b70c-a3ffc0123ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:18.707749 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15459702-9d7f-41db-b70c-a3ffc0123ef5 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:18.730625 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1315/2645] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:56:18.732223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757728df-82de-4fc5-991b-a6374081bc50 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:18.750593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757728df-82de-4fc5-991b-a6374081bc50 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:18.772161 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737916b-d36a-49cc-bca0-a5f59a39702f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:18.773284 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737916b-d36a-49cc-bca0-a5f59a39702f admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:18.773573 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737916b-d36a-49cc-bca0-a5f59a39702f admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:18.773853 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737916b-d36a-49cc-bca0-a5f59a39702f admin admin] PATH_INFO: `/v3/projects/aa0e7381c0db4bae9281c6a498efd913/users/bfd3c1f4f62641c0ba6557cbdc62e841/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:18.792211 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737916b-d36a-49cc-bca0-a5f59a39702f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa0e7381c0db4bae9281c6a498efd913, user_id=bfd3c1f4f62641c0ba6557cbdc62e841, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:18.795495 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2737916b-d36a-49cc-bca0-a5f59a39702f admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:18.815378 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1331/2646] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:56:18 2022] PUT /identity/v3/projects/aa0e7381c0db4bae9281c6a498efd913/users/bfd3c1f4f62641c0ba6557cbdc62e841/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:18.823895 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e354e-6528-400f-b500-f0de423e5d9e None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:18.826554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e354e-6528-400f-b500-f0de423e5d9e None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:18.828556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e354e-6528-400f-b500-f0de423e5d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:18.866607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-cd6e354e-6528-400f-b500-f0de423e5d9e None None] MFA Rules not processed for user `bfd3c1f4f62641c0ba6557cbdc62e841`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:18.868000 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-cd6e354e-6528-400f-b500-f0de423e5d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:18.964644 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1316/2647] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:18.975409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737916b-d36a-49cc-bca0-a5f59a39702f admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:18.977934 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-2737916b-d36a-49cc-bca0-a5f59a39702f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:19.053113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737916b-d36a-49cc-bca0-a5f59a39702f admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:19.058013 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd07c59-f668-45fe-a422-e408de44634e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:19.058884 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd07c59-f668-45fe-a422-e408de44634e service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:19.059216 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd07c59-f668-45fe-a422-e408de44634e service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:19.059676 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd07c59-f668-45fe-a422-e408de44634e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:19.069779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd07c59-f668-45fe-a422-e408de44634e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:19.071600 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd07c59-f668-45fe-a422-e408de44634e service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:19.082004 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1332/2648] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:56:18 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:56:21.075976 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e354e-6528-400f-b500-f0de423e5d9e None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:21.077598 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-cd6e354e-6528-400f-b500-f0de423e5d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:21.139697 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:21.143922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e354e-6528-400f-b500-f0de423e5d9e None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:21.149814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0ed7d-6c42-416d-8ac8-335ce80a88ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:21.151341 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0ed7d-6c42-416d-8ac8-335ce80a88ae service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:21.151985 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0ed7d-6c42-416d-8ac8-335ce80a88ae service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:21.152700 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0ed7d-6c42-416d-8ac8-335ce80a88ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:21.156619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-86f0ed7d-6c42-416d-8ac8-335ce80a88ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:21.172705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0ed7d-6c42-416d-8ac8-335ce80a88ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:21.174543 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0ed7d-6c42-416d-8ac8-335ce80a88ae service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:21.186718 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1317/2649] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:56:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:56:27.960749 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd07c59-f668-45fe-a422-e408de44634e service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:28.038464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:28.045659 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd07c59-f668-45fe-a422-e408de44634e service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:28.052450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51181b-cdf5-4f9b-a74a-150b5ec8de8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:28.053442 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51181b-cdf5-4f9b-a74a-150b5ec8de8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:28.053806 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51181b-cdf5-4f9b-a74a-150b5ec8de8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:28.054154 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51181b-cdf5-4f9b-a74a-150b5ec8de8b admin admin] PATH_INFO: `/v3/users/bfd3c1f4f62641c0ba6557cbdc62e841` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:28.055506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51181b-cdf5-4f9b-a74a-150b5ec8de8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfd3c1f4f62641c0ba6557cbdc62e841)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:28.058883 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51181b-cdf5-4f9b-a74a-150b5ec8de8b admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:28.119001 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-1e51181b-cdf5-4f9b-a74a-150b5ec8de8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfd3c1f4f62641c0ba6557cbdc62e841'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:28.136664 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-1e51181b-cdf5-4f9b-a74a-150b5ec8de8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfd3c1f4f62641c0ba6557cbdc62e841'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:28.146696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-1e51181b-cdf5-4f9b-a74a-150b5ec8de8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfd3c1f4f62641c0ba6557cbdc62e841'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:28.158914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-1e51181b-cdf5-4f9b-a74a-150b5ec8de8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfd3c1f4f62641c0ba6557cbdc62e841'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:28.159909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-1e51181b-cdf5-4f9b-a74a-150b5ec8de8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfd3c1f4f62641c0ba6557cbdc62e841'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:28.167369 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1333/2650] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:56:27 2022] DELETE /identity/v3/users/bfd3c1f4f62641c0ba6557cbdc62e841 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:28.172075 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bcdc6-ace5-44d7-a3fa-a8716428aaff None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:28.172824 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bcdc6-ace5-44d7-a3fa-a8716428aaff None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:28.172824 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bcdc6-ace5-44d7-a3fa-a8716428aaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:28.199586 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-052bcdc6-ace5-44d7-a3fa-a8716428aaff None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:28.200786 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-052bcdc6-ace5-44d7-a3fa-a8716428aaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:28.265974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1318/2651] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:28.275976 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51181b-cdf5-4f9b-a74a-150b5ec8de8b admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:28.278215 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-1e51181b-cdf5-4f9b-a74a-150b5ec8de8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:28.343394 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51181b-cdf5-4f9b-a74a-150b5ec8de8b admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:28.347909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca204e-47c6-4307-9636-b4f6e0617feb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:28.348823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca204e-47c6-4307-9636-b4f6e0617feb service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:28.349172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca204e-47c6-4307-9636-b4f6e0617feb service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:28.349528 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca204e-47c6-4307-9636-b4f6e0617feb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:28.360254 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca204e-47c6-4307-9636-b4f6e0617feb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:28.361682 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca204e-47c6-4307-9636-b4f6e0617feb service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:28.371390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1334/2652] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:56:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:56:28.659146 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bcdc6-ace5-44d7-a3fa-a8716428aaff None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:28.661225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-052bcdc6-ace5-44d7-a3fa-a8716428aaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:28.673854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bcdc6-ace5-44d7-a3fa-a8716428aaff None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:28.678546 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1d570-7c08-4431-8e48-82efe8f17b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:28.679467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1d570-7c08-4431-8e48-82efe8f17b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:28.679743 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1d570-7c08-4431-8e48-82efe8f17b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:28.680024 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1d570-7c08-4431-8e48-82efe8f17b1b admin admin] PATH_INFO: `/v3/projects/aa0e7381c0db4bae9281c6a498efd913` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:28.681225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1d570-7c08-4431-8e48-82efe8f17b1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa0e7381c0db4bae9281c6a498efd913)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:28.683697 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c1d570-7c08-4431-8e48-82efe8f17b1b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:28.730693 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-76c1d570-7c08-4431-8e48-82efe8f17b1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa0e7381c0db4bae9281c6a498efd913', 'request_id': 'req-76c1d570-7c08-4431-8e48-82efe8f17b1b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:28.737155 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-76c1d570-7c08-4431-8e48-82efe8f17b1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa0e7381c0db4bae9281c6a498efd913', 'request_id': 'req-76c1d570-7c08-4431-8e48-82efe8f17b1b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:28.743410 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-76c1d570-7c08-4431-8e48-82efe8f17b1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa0e7381c0db4bae9281c6a498efd913', 'request_id': 'req-76c1d570-7c08-4431-8e48-82efe8f17b1b'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:28.754285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-76c1d570-7c08-4431-8e48-82efe8f17b1b admin admin] The token cache is being invalidate because project aa0e7381c0db4bae9281c6a498efd913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:56:28.754524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-76c1d570-7c08-4431-8e48-82efe8f17b1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:28.755159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-76c1d570-7c08-4431-8e48-82efe8f17b1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:28.756730 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1319/2653] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:56:28 2022] DELETE /identity/v3/projects/aa0e7381c0db4bae9281c6a498efd913 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:28.762350 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ddcab3-4b55-4324-b661-2ab898f24cba None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:28.762646 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ddcab3-4b55-4324-b661-2ab898f24cba None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:28.762929 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ddcab3-4b55-4324-b661-2ab898f24cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:28.781312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-64ddcab3-4b55-4324-b661-2ab898f24cba None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:28.782530 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-64ddcab3-4b55-4324-b661-2ab898f24cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:28.841897 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1335/2654] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:28.849769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd460625-53cb-44fb-a57b-f23a83d4ed59 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:28.850098 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd460625-53cb-44fb-a57b-f23a83d4ed59 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:28.850384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd460625-53cb-44fb-a57b-f23a83d4ed59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:28.868597 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-dd460625-53cb-44fb-a57b-f23a83d4ed59 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:28.870328 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-dd460625-53cb-44fb-a57b-f23a83d4ed59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:28.878553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1320/2655] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:28.882858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ddcab3-4b55-4324-b661-2ab898f24cba None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:28.893810 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ddcab3-4b55-4324-b661-2ab898f24cba None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:28.898422 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788aaae-9765-4bee-98bb-53f870bb67e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:28.899069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788aaae-9765-4bee-98bb-53f870bb67e9 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:28.899376 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788aaae-9765-4bee-98bb-53f870bb67e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:28.899670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788aaae-9765-4bee-98bb-53f870bb67e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:28.900486 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788aaae-9765-4bee-98bb-53f870bb67e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:56:28.900798 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788aaae-9765-4bee-98bb-53f870bb67e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:28.902394 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788aaae-9765-4bee-98bb-53f870bb67e9 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:28.913884 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1336/2656] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:56:28.918525 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd460625-53cb-44fb-a57b-f23a83d4ed59 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:28.928541 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd460625-53cb-44fb-a57b-f23a83d4ed59 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:28.934077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d38cf-9e9f-4763-b817-4948febea4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:28.934741 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d38cf-9e9f-4763-b817-4948febea4e9 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:28.935026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d38cf-9e9f-4763-b817-4948febea4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:28.935323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d38cf-9e9f-4763-b817-4948febea4e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:28.936059 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d38cf-9e9f-4763-b817-4948febea4e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:28.938001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d38cf-9e9f-4763-b817-4948febea4e9 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:28.965787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1321/2657] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:56:28 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:56:28.969619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788aaae-9765-4bee-98bb-53f870bb67e9 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:28.976108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:28.980769 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788aaae-9765-4bee-98bb-53f870bb67e9 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:29.039962 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c1cb5-db5b-4147-9f43-7d85160e4304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:29.040621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c1cb5-db5b-4147-9f43-7d85160e4304 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:29.040903 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c1cb5-db5b-4147-9f43-7d85160e4304 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:29.041179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c1cb5-db5b-4147-9f43-7d85160e4304 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:29.042368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c1cb5-db5b-4147-9f43-7d85160e4304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:29.044303 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c1cb5-db5b-4147-9f43-7d85160e4304 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:29.064421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1337/2658] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:56:28 2022] POST /identity/v3/users => generated 302 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:56:29.068399 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d38cf-9e9f-4763-b817-4948febea4e9 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:29.078730 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d38cf-9e9f-4763-b817-4948febea4e9 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:29.083286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461e644-bbc1-4005-9985-228a30717eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:29.083963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461e644-bbc1-4005-9985-228a30717eca admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:29.084244 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461e644-bbc1-4005-9985-228a30717eca admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:29.084518 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461e644-bbc1-4005-9985-228a30717eca admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:29.085463 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461e644-bbc1-4005-9985-228a30717eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:29.087036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461e644-bbc1-4005-9985-228a30717eca admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:29.097722 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1322/2659] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:56:29.101596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c1cb5-db5b-4147-9f43-7d85160e4304 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:29.112317 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c1cb5-db5b-4147-9f43-7d85160e4304 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:29.117181 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58654d-29b3-4972-b480-afe428c2d4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:29.117591 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58654d-29b3-4972-b480-afe428c2d4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:29.117875 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58654d-29b3-4972-b480-afe428c2d4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:29.118153 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58654d-29b3-4972-b480-afe428c2d4f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:29.118869 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58654d-29b3-4972-b480-afe428c2d4f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:29.121105 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58654d-29b3-4972-b480-afe428c2d4f1 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:29.131054 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1338/2660] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:56:29.135118 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461e644-bbc1-4005-9985-228a30717eca admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:29.145347 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461e644-bbc1-4005-9985-228a30717eca admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:29.150062 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7e181-3d01-4f82-8c68-7cce75cab0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:29.150965 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7e181-3d01-4f82-8c68-7cce75cab0cc admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:29.151276 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7e181-3d01-4f82-8c68-7cce75cab0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:29.151624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7e181-3d01-4f82-8c68-7cce75cab0cc admin admin] PATH_INFO: `/v3/projects/980452db473946928605465df524df73/users/9ba51245602c44c6b8ef1b39d7031ec2/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:29.165986 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7e181-3d01-4f82-8c68-7cce75cab0cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=980452db473946928605465df524df73, user_id=9ba51245602c44c6b8ef1b39d7031ec2, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:29.169141 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7e181-3d01-4f82-8c68-7cce75cab0cc admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:29.181739 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1323/2661] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:56:29 2022] PUT /identity/v3/projects/980452db473946928605465df524df73/users/9ba51245602c44c6b8ef1b39d7031ec2/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:29.194191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0cd51-6607-431d-9224-7d98814e0697 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:29.194563 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0cd51-6607-431d-9224-7d98814e0697 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:29.194848 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0cd51-6607-431d-9224-7d98814e0697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:29.224369 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-67f0cd51-6607-431d-9224-7d98814e0697 None None] MFA Rules not processed for user `9ba51245602c44c6b8ef1b39d7031ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:29.225587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-67f0cd51-6607-431d-9224-7d98814e0697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:29.291972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1339/2662] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:29.296513 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7e181-3d01-4f82-8c68-7cce75cab0cc admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:29.309363 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:29.366261 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7e181-3d01-4f82-8c68-7cce75cab0cc admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:29.370927 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8f0b6-25a6-4259-adc1-94a236f8a664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:29.371620 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8f0b6-25a6-4259-adc1-94a236f8a664 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:29.371931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8f0b6-25a6-4259-adc1-94a236f8a664 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:29.372211 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8f0b6-25a6-4259-adc1-94a236f8a664 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:29.372894 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8f0b6-25a6-4259-adc1-94a236f8a664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:29.374795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8f0b6-25a6-4259-adc1-94a236f8a664 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:29.402212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1324/2663] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:56:29 2022] POST /identity/v3/projects => generated 405 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:56:29.406245 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0cd51-6607-431d-9224-7d98814e0697 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:29.418124 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0cd51-6607-431d-9224-7d98814e0697 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:29.478784 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fcaae-21d5-4a8e-9982-7147155f8a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:29.479548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fcaae-21d5-4a8e-9982-7147155f8a52 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:29.479832 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fcaae-21d5-4a8e-9982-7147155f8a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:29.480107 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fcaae-21d5-4a8e-9982-7147155f8a52 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:29.480824 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fcaae-21d5-4a8e-9982-7147155f8a52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:29.483953 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fcaae-21d5-4a8e-9982-7147155f8a52 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:29.503244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1340/2664] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:56:29 2022] POST /identity/v3/users => generated 309 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:56:29.507421 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8f0b6-25a6-4259-adc1-94a236f8a664 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:29.518802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8f0b6-25a6-4259-adc1-94a236f8a664 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:29.523609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e8e7d-88ad-4be5-8723-800f8b3a28da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:29.524311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e8e7d-88ad-4be5-8723-800f8b3a28da admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:29.524595 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e8e7d-88ad-4be5-8723-800f8b3a28da admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:29.524903 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6e8e7d-88ad-4be5-8723-800f8b3a28da admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:29.525630 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e8e7d-88ad-4be5-8723-800f8b3a28da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:29.527801 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6e8e7d-88ad-4be5-8723-800f8b3a28da admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:29.539121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1325/2665] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:56:29.544265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fcaae-21d5-4a8e-9982-7147155f8a52 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:29.549678 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:29.556440 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fcaae-21d5-4a8e-9982-7147155f8a52 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:29.561558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45dcdf6-dc31-4600-887b-d820ecb7cdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:29.562863 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45dcdf6-dc31-4600-887b-d820ecb7cdf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:29.563220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45dcdf6-dc31-4600-887b-d820ecb7cdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:29.563611 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45dcdf6-dc31-4600-887b-d820ecb7cdf4 admin admin] PATH_INFO: `/v3/projects/af2786ef8ab841d797dbb2ef2d9846ae/users/a5a198a3d41442c4b8d963f118513cdb/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:29.579801 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45dcdf6-dc31-4600-887b-d820ecb7cdf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af2786ef8ab841d797dbb2ef2d9846ae, user_id=a5a198a3d41442c4b8d963f118513cdb, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:29.582465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45dcdf6-dc31-4600-887b-d820ecb7cdf4 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:29.596743 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1341/2666] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:56:29 2022] PUT /identity/v3/projects/af2786ef8ab841d797dbb2ef2d9846ae/users/a5a198a3d41442c4b8d963f118513cdb/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:29.600912 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e8e7d-88ad-4be5-8723-800f8b3a28da admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:29.611501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6e8e7d-88ad-4be5-8723-800f8b3a28da admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:29.671600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8c1a1-6b1d-4511-8a03-a821683db241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:29.672280 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8c1a1-6b1d-4511-8a03-a821683db241 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:29.672664 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8c1a1-6b1d-4511-8a03-a821683db241 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:29.672829 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8c1a1-6b1d-4511-8a03-a821683db241 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:29.673561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef8c1a1-6b1d-4511-8a03-a821683db241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:29.675152 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef8c1a1-6b1d-4511-8a03-a821683db241 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:29.685781 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1326/2667] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:56:29.690087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45dcdf6-dc31-4600-887b-d820ecb7cdf4 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:29.702923 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45dcdf6-dc31-4600-887b-d820ecb7cdf4 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:29.716661 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c072f9-f703-40f7-8489-8f91df356a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:29.717592 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c072f9-f703-40f7-8489-8f91df356a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:29.718091 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c072f9-f703-40f7-8489-8f91df356a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:29.718379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c072f9-f703-40f7-8489-8f91df356a1c admin admin] PATH_INFO: `/v3/projects/af2786ef8ab841d797dbb2ef2d9846ae/users/a5a198a3d41442c4b8d963f118513cdb/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:29.723081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c072f9-f703-40f7-8489-8f91df356a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=af2786ef8ab841d797dbb2ef2d9846ae, user_id=a5a198a3d41442c4b8d963f118513cdb, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:29.725820 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c072f9-f703-40f7-8489-8f91df356a1c admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:29.738029 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1342/2668] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:56:29 2022] PUT /identity/v3/projects/af2786ef8ab841d797dbb2ef2d9846ae/users/a5a198a3d41442c4b8d963f118513cdb/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:29.748806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31cf22-6be6-43c3-90d4-a6b5f962d810 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:29.748806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31cf22-6be6-43c3-90d4-a6b5f962d810 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:29.748806 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31cf22-6be6-43c3-90d4-a6b5f962d810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:29.786594 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-fd31cf22-6be6-43c3-90d4-a6b5f962d810 None None] MFA Rules not processed for user `a5a198a3d41442c4b8d963f118513cdb`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:29.788420 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-fd31cf22-6be6-43c3-90d4-a6b5f962d810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:29.867397 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1327/2669] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:29 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:29.878213 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c072f9-f703-40f7-8489-8f91df356a1c admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:29.880066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-17c072f9-f703-40f7-8489-8f91df356a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:29.942737 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c072f9-f703-40f7-8489-8f91df356a1c admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:29.946760 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61df933-21fb-4d24-97a1-5c90a7dd2688 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:29.947440 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61df933-21fb-4d24-97a1-5c90a7dd2688 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:29.948296 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61df933-21fb-4d24-97a1-5c90a7dd2688 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:29.948568 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61df933-21fb-4d24-97a1-5c90a7dd2688 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:29.958064 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61df933-21fb-4d24-97a1-5c90a7dd2688 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:29.959533 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61df933-21fb-4d24-97a1-5c90a7dd2688 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:29.969525 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1343/2670] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:56:29 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:56:30.114926 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31cf22-6be6-43c3-90d4-a6b5f962d810 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:30.124531 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:30.183058 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31cf22-6be6-43c3-90d4-a6b5f962d810 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:30.190738 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e1088-1a66-43bd-9c88-a049eb2094bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:30.192278 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e1088-1a66-43bd-9c88-a049eb2094bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:30.192839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e1088-1a66-43bd-9c88-a049eb2094bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:30.193331 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e1088-1a66-43bd-9c88-a049eb2094bd admin admin] PATH_INFO: `/v3/users/9ba51245602c44c6b8ef1b39d7031ec2` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:30.195500 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e1088-1a66-43bd-9c88-a049eb2094bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ba51245602c44c6b8ef1b39d7031ec2)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:30.199967 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e1088-1a66-43bd-9c88-a049eb2094bd admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:30.249522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-950e1088-1a66-43bd-9c88-a049eb2094bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ba51245602c44c6b8ef1b39d7031ec2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:30.261954 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-950e1088-1a66-43bd-9c88-a049eb2094bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ba51245602c44c6b8ef1b39d7031ec2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:30.269281 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-950e1088-1a66-43bd-9c88-a049eb2094bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ba51245602c44c6b8ef1b39d7031ec2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:30.270173 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-950e1088-1a66-43bd-9c88-a049eb2094bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ba51245602c44c6b8ef1b39d7031ec2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:30.277233 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-950e1088-1a66-43bd-9c88-a049eb2094bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ba51245602c44c6b8ef1b39d7031ec2'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:30.283280 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1328/2671] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:56:30 2022] DELETE /identity/v3/users/9ba51245602c44c6b8ef1b39d7031ec2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:30.287785 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3743f-0a76-4515-8fc8-4f899734dd04 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:30.288268 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3743f-0a76-4515-8fc8-4f899734dd04 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:30.288384 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3743f-0a76-4515-8fc8-4f899734dd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:30.306029 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d2e3743f-0a76-4515-8fc8-4f899734dd04 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:30.307235 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d2e3743f-0a76-4515-8fc8-4f899734dd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:30.365945 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1344/2672] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:30.376078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e1088-1a66-43bd-9c88-a049eb2094bd admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:30.377471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-950e1088-1a66-43bd-9c88-a049eb2094bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:30.436495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e1088-1a66-43bd-9c88-a049eb2094bd admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:30.440247 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c2d60-959f-42f7-aa9c-4401fbceda14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:30.440892 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c2d60-959f-42f7-aa9c-4401fbceda14 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:30.441179 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c2d60-959f-42f7-aa9c-4401fbceda14 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:30.441460 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c2d60-959f-42f7-aa9c-4401fbceda14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:30.451331 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c2d60-959f-42f7-aa9c-4401fbceda14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:30.452795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c2d60-959f-42f7-aa9c-4401fbceda14 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:30.462373 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1329/2673] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:56:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:56:30.725602 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3743f-0a76-4515-8fc8-4f899734dd04 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:30.727040 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d2e3743f-0a76-4515-8fc8-4f899734dd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:30.741112 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3743f-0a76-4515-8fc8-4f899734dd04 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:30.746457 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6f6ae-19f2-463e-b6b0-1f751532a810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:30.747516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6f6ae-19f2-463e-b6b0-1f751532a810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:30.747801 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6f6ae-19f2-463e-b6b0-1f751532a810 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:30.748077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6f6ae-19f2-463e-b6b0-1f751532a810 admin admin] PATH_INFO: `/v3/projects/980452db473946928605465df524df73` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:30.749266 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6f6ae-19f2-463e-b6b0-1f751532a810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=980452db473946928605465df524df73)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:30.751156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6f6ae-19f2-463e-b6b0-1f751532a810 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:30.799237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f9b6f6ae-19f2-463e-b6b0-1f751532a810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '980452db473946928605465df524df73', 'request_id': 'req-f9b6f6ae-19f2-463e-b6b0-1f751532a810'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:30.806333 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f9b6f6ae-19f2-463e-b6b0-1f751532a810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '980452db473946928605465df524df73', 'request_id': 'req-f9b6f6ae-19f2-463e-b6b0-1f751532a810'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:30.813500 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f9b6f6ae-19f2-463e-b6b0-1f751532a810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '980452db473946928605465df524df73', 'request_id': 'req-f9b6f6ae-19f2-463e-b6b0-1f751532a810'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:30.815511 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:30.826297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f9b6f6ae-19f2-463e-b6b0-1f751532a810 admin admin] The token cache is being invalidate because project 980452db473946928605465df524df73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:56:30.826587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f9b6f6ae-19f2-463e-b6b0-1f751532a810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:30.827225 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f9b6f6ae-19f2-463e-b6b0-1f751532a810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:30.828795 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1345/2674] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:56:30 2022] DELETE /identity/v3/projects/980452db473946928605465df524df73 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:30.832704 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c2d60-959f-42f7-aa9c-4401fbceda14 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:30.842923 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c2d60-959f-42f7-aa9c-4401fbceda14 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:30.870663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:30.903551 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03c388-54b7-4820-8d05-cd5abb04a9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:30.904615 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03c388-54b7-4820-8d05-cd5abb04a9d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:30.904834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03c388-54b7-4820-8d05-cd5abb04a9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:30.907769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03c388-54b7-4820-8d05-cd5abb04a9d0 admin admin] PATH_INFO: `/v3/users/a5a198a3d41442c4b8d963f118513cdb` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:30.908435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03c388-54b7-4820-8d05-cd5abb04a9d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5a198a3d41442c4b8d963f118513cdb)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:30.910380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03c388-54b7-4820-8d05-cd5abb04a9d0 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:30.963559 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-fe03c388-54b7-4820-8d05-cd5abb04a9d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5a198a3d41442c4b8d963f118513cdb'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:30.976487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-fe03c388-54b7-4820-8d05-cd5abb04a9d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5a198a3d41442c4b8d963f118513cdb'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:30.984096 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-fe03c388-54b7-4820-8d05-cd5abb04a9d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5a198a3d41442c4b8d963f118513cdb'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:30.984711 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-fe03c388-54b7-4820-8d05-cd5abb04a9d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5a198a3d41442c4b8d963f118513cdb'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:30.992221 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-fe03c388-54b7-4820-8d05-cd5abb04a9d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5a198a3d41442c4b8d963f118513cdb'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:30.999446 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1330/2675] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:56:30 2022] DELETE /identity/v3/users/a5a198a3d41442c4b8d963f118513cdb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:31.271360 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6f6ae-19f2-463e-b6b0-1f751532a810 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:31.273032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f9b6f6ae-19f2-463e-b6b0-1f751532a810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:31.287515 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6f6ae-19f2-463e-b6b0-1f751532a810 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:31.348981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3681f-2ada-40e0-958a-ff6415d4806d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:31.349834 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3681f-2ada-40e0-958a-ff6415d4806d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:31.350114 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3681f-2ada-40e0-958a-ff6415d4806d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:31.350392 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3681f-2ada-40e0-958a-ff6415d4806d admin admin] PATH_INFO: `/v3/projects/af2786ef8ab841d797dbb2ef2d9846ae` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:31.351613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3681f-2ada-40e0-958a-ff6415d4806d admin admin] RBAC: Authorizing `identity:delete_project(project_id=af2786ef8ab841d797dbb2ef2d9846ae)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:31.353512 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3681f-2ada-40e0-958a-ff6415d4806d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:31.403893 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-47a3681f-2ada-40e0-958a-ff6415d4806d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af2786ef8ab841d797dbb2ef2d9846ae', 'request_id': 'req-47a3681f-2ada-40e0-958a-ff6415d4806d'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:31.410013 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-47a3681f-2ada-40e0-958a-ff6415d4806d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af2786ef8ab841d797dbb2ef2d9846ae', 'request_id': 'req-47a3681f-2ada-40e0-958a-ff6415d4806d'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:31.417307 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-47a3681f-2ada-40e0-958a-ff6415d4806d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af2786ef8ab841d797dbb2ef2d9846ae', 'request_id': 'req-47a3681f-2ada-40e0-958a-ff6415d4806d'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:31.427830 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-47a3681f-2ada-40e0-958a-ff6415d4806d admin admin] The token cache is being invalidate because project af2786ef8ab841d797dbb2ef2d9846ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:56:31.428119 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-47a3681f-2ada-40e0-958a-ff6415d4806d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:31.428757 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-47a3681f-2ada-40e0-958a-ff6415d4806d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:31.430327 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1346/2676] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:56:31 2022] DELETE /identity/v3/projects/af2786ef8ab841d797dbb2ef2d9846ae => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:31.435595 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20295412-8e6a-4dd3-b0b2-d5adc2b01422 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:31.436099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20295412-8e6a-4dd3-b0b2-d5adc2b01422 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:31.436209 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20295412-8e6a-4dd3-b0b2-d5adc2b01422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:31.457241 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-20295412-8e6a-4dd3-b0b2-d5adc2b01422 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:31.458556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-20295412-8e6a-4dd3-b0b2-d5adc2b01422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:31.522786 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1331/2677] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:31.531435 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985448f-a4bc-4484-82be-47ffd6a52a57 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:31.531726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985448f-a4bc-4484-82be-47ffd6a52a57 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:31.532006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985448f-a4bc-4484-82be-47ffd6a52a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:31.550771 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f985448f-a4bc-4484-82be-47ffd6a52a57 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:31.552053 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f985448f-a4bc-4484-82be-47ffd6a52a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:31.560113 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1347/2678] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:31.564251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20295412-8e6a-4dd3-b0b2-d5adc2b01422 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:31.574543 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20295412-8e6a-4dd3-b0b2-d5adc2b01422 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:31.579327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8090529-ab9b-446a-904d-59e2e5979143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:31.580018 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8090529-ab9b-446a-904d-59e2e5979143 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:31.580303 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8090529-ab9b-446a-904d-59e2e5979143 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:31.580580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8090529-ab9b-446a-904d-59e2e5979143 admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:31.581305 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8090529-ab9b-446a-904d-59e2e5979143 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:56:31.581611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8090529-ab9b-446a-904d-59e2e5979143 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:31.583249 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8090529-ab9b-446a-904d-59e2e5979143 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:31.595680 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1332/2679] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:56:31.600042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985448f-a4bc-4484-82be-47ffd6a52a57 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:31.609267 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985448f-a4bc-4484-82be-47ffd6a52a57 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:31.614372 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab3088-5f87-4b35-92e9-c7d891abba12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:31.614663 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab3088-5f87-4b35-92e9-c7d891abba12 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:31.614945 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab3088-5f87-4b35-92e9-c7d891abba12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:31.615219 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab3088-5f87-4b35-92e9-c7d891abba12 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:31.615931 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab3088-5f87-4b35-92e9-c7d891abba12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:31.617812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab3088-5f87-4b35-92e9-c7d891abba12 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:31.647552 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1348/2680] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:56:31 2022] POST /identity/v3/projects => generated 403 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:56:31.648698 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8090529-ab9b-446a-904d-59e2e5979143 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:31.659563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8090529-ab9b-446a-904d-59e2e5979143 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:31.695339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:31.724407 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a78a75-d6b5-4811-9ac9-56908dce5200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:31.725080 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a78a75-d6b5-4811-9ac9-56908dce5200 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:31.725418 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a78a75-d6b5-4811-9ac9-56908dce5200 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:31.725620 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a78a75-d6b5-4811-9ac9-56908dce5200 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:31.726318 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a78a75-d6b5-4811-9ac9-56908dce5200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:31.728259 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a78a75-d6b5-4811-9ac9-56908dce5200 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:31.748595 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1333/2681] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:56:31 2022] POST /identity/v3/users => generated 302 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:56:31.752572 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab3088-5f87-4b35-92e9-c7d891abba12 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:31.763191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab3088-5f87-4b35-92e9-c7d891abba12 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:31.768375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790bf3c-587b-44df-b98f-158a6dabe5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:31.769030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790bf3c-587b-44df-b98f-158a6dabe5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:31.769342 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790bf3c-587b-44df-b98f-158a6dabe5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:31.769615 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790bf3c-587b-44df-b98f-158a6dabe5d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:31.770338 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790bf3c-587b-44df-b98f-158a6dabe5d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:31.772109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790bf3c-587b-44df-b98f-158a6dabe5d7 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:31.782991 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1349/2682] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:56:31.787702 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a78a75-d6b5-4811-9ac9-56908dce5200 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:31.799175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a78a75-d6b5-4811-9ac9-56908dce5200 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:31.805183 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5849e517-10da-4198-95fa-3ddfc4140fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:31.806128 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849e517-10da-4198-95fa-3ddfc4140fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:31.806542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849e517-10da-4198-95fa-3ddfc4140fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:31.806824 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5849e517-10da-4198-95fa-3ddfc4140fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:31.808101 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849e517-10da-4198-95fa-3ddfc4140fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:31.809710 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5849e517-10da-4198-95fa-3ddfc4140fa3 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:31.820554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1334/2683] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:56:31.825330 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790bf3c-587b-44df-b98f-158a6dabe5d7 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:31.836076 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790bf3c-587b-44df-b98f-158a6dabe5d7 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:31.840765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3eb8bc-274f-4c98-ba57-130f8538b313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:31.841674 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3eb8bc-274f-4c98-ba57-130f8538b313 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:31.841956 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3eb8bc-274f-4c98-ba57-130f8538b313 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:31.842248 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3eb8bc-274f-4c98-ba57-130f8538b313 admin admin] PATH_INFO: `/v3/projects/65277242f5d94a4e8da8b38bd4bd750a/users/b84b86462aa04aae98f72d3ba826c797/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:31.857724 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3eb8bc-274f-4c98-ba57-130f8538b313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65277242f5d94a4e8da8b38bd4bd750a, user_id=b84b86462aa04aae98f72d3ba826c797, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:31.860420 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3eb8bc-274f-4c98-ba57-130f8538b313 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:31.873364 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1350/2684] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:56:31 2022] PUT /identity/v3/projects/65277242f5d94a4e8da8b38bd4bd750a/users/b84b86462aa04aae98f72d3ba826c797/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:31.878430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bf2cb-c87d-4565-86f3-f3ab97456314 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:31.878752 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bf2cb-c87d-4565-86f3-f3ab97456314 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:31.879036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bf2cb-c87d-4565-86f3-f3ab97456314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:31.897486 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-778bf2cb-c87d-4565-86f3-f3ab97456314 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:31.898688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-778bf2cb-c87d-4565-86f3-f3ab97456314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:31.956983 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1335/2685] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:31.966597 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3eb8bc-274f-4c98-ba57-130f8538b313 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:31.968328 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-8d3eb8bc-274f-4c98-ba57-130f8538b313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:31.988743 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:32.028472 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3eb8bc-274f-4c98-ba57-130f8538b313 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:32.032437 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0efd9-f339-417f-9103-e9b88a1ee722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:32.033093 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0efd9-f339-417f-9103-e9b88a1ee722 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:32.033380 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0efd9-f339-417f-9103-e9b88a1ee722 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:32.033661 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d0efd9-f339-417f-9103-e9b88a1ee722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:32.045503 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0efd9-f339-417f-9103-e9b88a1ee722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:32.047162 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d0efd9-f339-417f-9103-e9b88a1ee722 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:32.057196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1351/2686] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:56:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:56:32.709024 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e11b23-794f-4c4a-acf3-7c8a02c538a7 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:32.709542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e11b23-794f-4c4a-acf3-7c8a02c538a7 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:32.709652 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e11b23-794f-4c4a-acf3-7c8a02c538a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:32.742050 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-b1e11b23-794f-4c4a-acf3-7c8a02c538a7 None None] MFA Rules not processed for user `b84b86462aa04aae98f72d3ba826c797`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:32.743429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b1e11b23-794f-4c4a-acf3-7c8a02c538a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:32.820127 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1336/2687] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:32.827059 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0efd9-f339-417f-9103-e9b88a1ee722 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:32.842759 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d0efd9-f339-417f-9103-e9b88a1ee722 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:32.848490 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195357d-1a7b-4169-90e4-b95b38027a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:32.849445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195357d-1a7b-4169-90e4-b95b38027a9f admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:32.849860 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195357d-1a7b-4169-90e4-b95b38027a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:32.850209 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195357d-1a7b-4169-90e4-b95b38027a9f admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:32.851023 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195357d-1a7b-4169-90e4-b95b38027a9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:32.853104 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195357d-1a7b-4169-90e4-b95b38027a9f admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:32.879161 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1352/2688] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:56:32 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:56:32.883177 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e11b23-794f-4c4a-acf3-7c8a02c538a7 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:32.896206 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e11b23-794f-4c4a-acf3-7c8a02c538a7 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:32.945350 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:32.963610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90a0e8-aa65-4fe2-9caf-8d14df327d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:32.964834 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90a0e8-aa65-4fe2-9caf-8d14df327d3d admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:32.965118 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90a0e8-aa65-4fe2-9caf-8d14df327d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:32.965392 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90a0e8-aa65-4fe2-9caf-8d14df327d3d admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:32.966097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90a0e8-aa65-4fe2-9caf-8d14df327d3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:32.968390 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90a0e8-aa65-4fe2-9caf-8d14df327d3d admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:32.991858 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1337/2689] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:56:32 2022] POST /identity/v3/users => generated 308 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:56:32.993174 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195357d-1a7b-4169-90e4-b95b38027a9f admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:33.003442 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195357d-1a7b-4169-90e4-b95b38027a9f admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:33.008301 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf46675-1a53-4b57-8087-2b87006553f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:33.008959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf46675-1a53-4b57-8087-2b87006553f7 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:33.009244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf46675-1a53-4b57-8087-2b87006553f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:33.009520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf46675-1a53-4b57-8087-2b87006553f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:33.010260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf46675-1a53-4b57-8087-2b87006553f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:33.012762 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf46675-1a53-4b57-8087-2b87006553f7 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:33.028838 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1353/2690] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:56:33.030351 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90a0e8-aa65-4fe2-9caf-8d14df327d3d admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:33.040422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90a0e8-aa65-4fe2-9caf-8d14df327d3d admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:33.045810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d423a00-3b21-483f-9569-6fe85817e124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:33.046686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d423a00-3b21-483f-9569-6fe85817e124 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:33.047005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d423a00-3b21-483f-9569-6fe85817e124 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:33.047282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d423a00-3b21-483f-9569-6fe85817e124 admin admin] PATH_INFO: `/v3/projects/5e9e1d963a544a2bb59c49c067a4b797/users/3f4b755b29724a9b961493cb78e8c2d9/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:33.062414 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d423a00-3b21-483f-9569-6fe85817e124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e9e1d963a544a2bb59c49c067a4b797, user_id=3f4b755b29724a9b961493cb78e8c2d9, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:33.065098 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d423a00-3b21-483f-9569-6fe85817e124 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:33.080210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1338/2691] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:56:33 2022] PUT /identity/v3/projects/5e9e1d963a544a2bb59c49c067a4b797/users/3f4b755b29724a9b961493cb78e8c2d9/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:33.084294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf46675-1a53-4b57-8087-2b87006553f7 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:33.094961 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf46675-1a53-4b57-8087-2b87006553f7 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:33.162625 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f37ce-d8e6-4119-bb49-bc2afb3b408f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:33.163875 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f37ce-d8e6-4119-bb49-bc2afb3b408f admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:33.164213 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f37ce-d8e6-4119-bb49-bc2afb3b408f admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:33.164551 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f37ce-d8e6-4119-bb49-bc2afb3b408f admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:33.165321 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f37ce-d8e6-4119-bb49-bc2afb3b408f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:33.167222 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f37ce-d8e6-4119-bb49-bc2afb3b408f admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:33.168201 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb6f00-c51a-42f2-8045-d64d5244e276 None None] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:33.168498 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb6f00-c51a-42f2-8045-d64d5244e276 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:33.168780 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb6f00-c51a-42f2-8045-d64d5244e276 None None] PATH_INFO: `/` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:33.170155 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1339/2692] 10.222.0.36 () {54 vars in 921 bytes} [Wed Jun 22 11:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 22 11:56:33.175220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53194f30-34b7-4b02-90b7-e9843d993cd5 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:33.175554 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53194f30-34b7-4b02-90b7-e9843d993cd5 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:33.175874 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53194f30-34b7-4b02-90b7-e9843d993cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:33.179695 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1354/2693] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:56:33.184548 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f37ce-d8e6-4119-bb49-bc2afb3b408f admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:33.194738 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f37ce-d8e6-4119-bb49-bc2afb3b408f admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:33.201449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797113e2-53ba-443c-a0fa-54dad250be4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:33.202367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797113e2-53ba-443c-a0fa-54dad250be4c admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:33.202657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797113e2-53ba-443c-a0fa-54dad250be4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:33.202937 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797113e2-53ba-443c-a0fa-54dad250be4c admin admin] PATH_INFO: `/v3/projects/5e9e1d963a544a2bb59c49c067a4b797/users/3f4b755b29724a9b961493cb78e8c2d9/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:33.205120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797113e2-53ba-443c-a0fa-54dad250be4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e9e1d963a544a2bb59c49c067a4b797, user_id=3f4b755b29724a9b961493cb78e8c2d9, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:33.207971 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797113e2-53ba-443c-a0fa-54dad250be4c admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:33.210332 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-53194f30-34b7-4b02-90b7-e9843d993cd5 None None] MFA Rules not processed for user `479a376dcac14c509c1c55f0046d88cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:33.211808 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-53194f30-34b7-4b02-90b7-e9843d993cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:33.219520 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1355/2694] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:56:33 2022] PUT /identity/v3/projects/5e9e1d963a544a2bb59c49c067a4b797/users/3f4b755b29724a9b961493cb78e8c2d9/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:33.287624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1340/2695] 10.222.0.36 () {58 vars in 1018 bytes} [Wed Jun 22 11:56:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:56:33.298919 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797113e2-53ba-443c-a0fa-54dad250be4c admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:33.300865 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-797113e2-53ba-443c-a0fa-54dad250be4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:33.371217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797113e2-53ba-443c-a0fa-54dad250be4c admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:33.374745 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2408b-cd46-4e5a-b1d5-468ab4926825 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ed133721ed044708ee362c93f564d07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:33.375425 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2408b-cd46-4e5a-b1d5-468ab4926825 service glance] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:33.375712 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2408b-cd46-4e5a-b1d5-468ab4926825 service glance] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:33.375988 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2408b-cd46-4e5a-b1d5-468ab4926825 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:33.387559 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2408b-cd46-4e5a-b1d5-468ab4926825 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:33.389074 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2408b-cd46-4e5a-b1d5-468ab4926825 service glance] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:33.400940 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1356/2696] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:56:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:56:33.611974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53194f30-34b7-4b02-90b7-e9843d993cd5 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:33.613400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-53194f30-34b7-4b02-90b7-e9843d993cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:33.691238 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53194f30-34b7-4b02-90b7-e9843d993cd5 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:33.696840 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1577c4-5089-4f78-8ca0-a146b08997e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93630ed5e6dc4a5f958bc7cf1a68a26c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:33.697495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1577c4-5089-4f78-8ca0-a146b08997e8 service placement] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:33.697776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1577c4-5089-4f78-8ca0-a146b08997e8 service placement] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:33.698083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1577c4-5089-4f78-8ca0-a146b08997e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:33.699988 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-4e1577c4-5089-4f78-8ca0-a146b08997e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:33.712297 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1577c4-5089-4f78-8ca0-a146b08997e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:33.714394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1577c4-5089-4f78-8ca0-a146b08997e8 service placement] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:33.726550 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1341/2697] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:56:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:56:34.000683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbb5cf-f62f-4075-816d-372dff14c2ed None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:34.001220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbb5cf-f62f-4075-816d-372dff14c2ed None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:34.001804 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbb5cf-f62f-4075-816d-372dff14c2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:34.030044 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:34.034222 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-29cbb5cf-f62f-4075-816d-372dff14c2ed None None] MFA Rules not processed for user `3f4b755b29724a9b961493cb78e8c2d9`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:34.036166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-29cbb5cf-f62f-4075-816d-372dff14c2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:34.130694 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1357/2698] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:33 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:35.355530 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1577c4-5089-4f78-8ca0-a146b08997e8 service placement] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:35.391371 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:35.422191 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1577c4-5089-4f78-8ca0-a146b08997e8 service placement] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:35.426743 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb22bd1-bf19-48e0-98b9-1668dddeb121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:35.427650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb22bd1-bf19-48e0-98b9-1668dddeb121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:35.427934 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb22bd1-bf19-48e0-98b9-1668dddeb121 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:35.428212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb22bd1-bf19-48e0-98b9-1668dddeb121 admin admin] PATH_INFO: `/v3/users/b84b86462aa04aae98f72d3ba826c797` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:35.429563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb22bd1-bf19-48e0-98b9-1668dddeb121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b84b86462aa04aae98f72d3ba826c797)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:35.431543 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb22bd1-bf19-48e0-98b9-1668dddeb121 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:35.485222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cdb22bd1-bf19-48e0-98b9-1668dddeb121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b84b86462aa04aae98f72d3ba826c797'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:35.498654 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cdb22bd1-bf19-48e0-98b9-1668dddeb121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b84b86462aa04aae98f72d3ba826c797'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:35.508016 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cdb22bd1-bf19-48e0-98b9-1668dddeb121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b84b86462aa04aae98f72d3ba826c797'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:35.508921 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cdb22bd1-bf19-48e0-98b9-1668dddeb121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b84b86462aa04aae98f72d3ba826c797'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:35.517894 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cdb22bd1-bf19-48e0-98b9-1668dddeb121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b84b86462aa04aae98f72d3ba826c797'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:35.524821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefb66e-26ee-49f1-89c6-501ec85a7c61 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:35.525305 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefb66e-26ee-49f1-89c6-501ec85a7c61 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:35.525423 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefb66e-26ee-49f1-89c6-501ec85a7c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:35.528397 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1342/2699] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:56:35 2022] DELETE /identity/v3/users/b84b86462aa04aae98f72d3ba826c797 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:35.556615 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-cfefb66e-26ee-49f1-89c6-501ec85a7c61 None None] MFA Rules not processed for user `955c44ffe1bd426e900c7405df44d1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:35.557866 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-cfefb66e-26ee-49f1-89c6-501ec85a7c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:35.626690 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1358/2700] 10.222.0.36 () {58 vars in 1009 bytes} [Wed Jun 22 11:56:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:56:35.640814 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb22bd1-bf19-48e0-98b9-1668dddeb121 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:35.642277 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-cdb22bd1-bf19-48e0-98b9-1668dddeb121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:35.717192 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb22bd1-bf19-48e0-98b9-1668dddeb121 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:35.722359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4f1b5-c60b-4628-a14f-dbbeedce18ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd642e290e44e67b56eb3b8361d7af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:35.723022 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4f1b5-c60b-4628-a14f-dbbeedce18ca service swift] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:35.723301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4f1b5-c60b-4628-a14f-dbbeedce18ca service swift] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:35.723601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4f1b5-c60b-4628-a14f-dbbeedce18ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:35.734793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4f1b5-c60b-4628-a14f-dbbeedce18ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:35.736785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4f1b5-c60b-4628-a14f-dbbeedce18ca service swift] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:35.746972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1343/2701] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jun 22 11:56:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 22 11:56:35.786395 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefb66e-26ee-49f1-89c6-501ec85a7c61 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:35.788045 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-cfefb66e-26ee-49f1-89c6-501ec85a7c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:35.885260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefb66e-26ee-49f1-89c6-501ec85a7c61 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:35.892639 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b41d0f-322f-4a62-b651-0957424bf692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:35.893527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b41d0f-322f-4a62-b651-0957424bf692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:35.893845 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b41d0f-322f-4a62-b651-0957424bf692 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:35.894108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b41d0f-322f-4a62-b651-0957424bf692 admin admin] PATH_INFO: `/v3/projects/65277242f5d94a4e8da8b38bd4bd750a` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:35.895749 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b41d0f-322f-4a62-b651-0957424bf692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65277242f5d94a4e8da8b38bd4bd750a)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:35.897900 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b41d0f-322f-4a62-b651-0957424bf692 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:35.991982 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:36.001988 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b9b41d0f-322f-4a62-b651-0957424bf692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65277242f5d94a4e8da8b38bd4bd750a', 'request_id': 'req-b9b41d0f-322f-4a62-b651-0957424bf692'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:36.025042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b9b41d0f-322f-4a62-b651-0957424bf692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65277242f5d94a4e8da8b38bd4bd750a', 'request_id': 'req-b9b41d0f-322f-4a62-b651-0957424bf692'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:36.044312 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b9b41d0f-322f-4a62-b651-0957424bf692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65277242f5d94a4e8da8b38bd4bd750a', 'request_id': 'req-b9b41d0f-322f-4a62-b651-0957424bf692'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:36.072906 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b9b41d0f-322f-4a62-b651-0957424bf692 admin admin] The token cache is being invalidate because project 65277242f5d94a4e8da8b38bd4bd750a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:56:36.073232 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b9b41d0f-322f-4a62-b651-0957424bf692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:36.074372 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b9b41d0f-322f-4a62-b651-0957424bf692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:36.081323 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1359/2702] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:56:35 2022] DELETE /identity/v3/projects/65277242f5d94a4e8da8b38bd4bd750a => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:36.082687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4f1b5-c60b-4628-a14f-dbbeedce18ca service swift] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:36.124796 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4f1b5-c60b-4628-a14f-dbbeedce18ca service swift] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:36.198488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:36.240767 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b3d07-a282-48c3-b800-79fdae3bcf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:36.243009 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b3d07-a282-48c3-b800-79fdae3bcf26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:36.243766 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b3d07-a282-48c3-b800-79fdae3bcf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:36.244509 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b3d07-a282-48c3-b800-79fdae3bcf26 admin admin] PATH_INFO: `/v3/users/3f4b755b29724a9b961493cb78e8c2d9` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:36.246447 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b3d07-a282-48c3-b800-79fdae3bcf26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f4b755b29724a9b961493cb78e8c2d9)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:36.248675 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b3d07-a282-48c3-b800-79fdae3bcf26 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:36.345722 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-968b3d07-a282-48c3-b800-79fdae3bcf26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f4b755b29724a9b961493cb78e8c2d9'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:36.364253 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-968b3d07-a282-48c3-b800-79fdae3bcf26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f4b755b29724a9b961493cb78e8c2d9'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:36.374035 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-968b3d07-a282-48c3-b800-79fdae3bcf26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f4b755b29724a9b961493cb78e8c2d9'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:36.374960 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-968b3d07-a282-48c3-b800-79fdae3bcf26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f4b755b29724a9b961493cb78e8c2d9'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:36.383606 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-968b3d07-a282-48c3-b800-79fdae3bcf26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f4b755b29724a9b961493cb78e8c2d9'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:36.394299 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1344/2703] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:56:36 2022] DELETE /identity/v3/users/3f4b755b29724a9b961493cb78e8c2d9 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:36.621648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b41d0f-322f-4a62-b651-0957424bf692 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:36.623927 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b9b41d0f-322f-4a62-b651-0957424bf692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:36.649798 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b41d0f-322f-4a62-b651-0957424bf692 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:36.721530 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d5d05-1578-4c53-b4c1-64cf2d73784e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:36.722799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d5d05-1578-4c53-b4c1-64cf2d73784e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:36.723188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d5d05-1578-4c53-b4c1-64cf2d73784e admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:36.723746 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d5d05-1578-4c53-b4c1-64cf2d73784e admin admin] PATH_INFO: `/v3/projects/5e9e1d963a544a2bb59c49c067a4b797` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:36.725818 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d5d05-1578-4c53-b4c1-64cf2d73784e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e9e1d963a544a2bb59c49c067a4b797)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:36.728445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d5d05-1578-4c53-b4c1-64cf2d73784e admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:36.827496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8d6d5d05-1578-4c53-b4c1-64cf2d73784e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e9e1d963a544a2bb59c49c067a4b797', 'request_id': 'req-8d6d5d05-1578-4c53-b4c1-64cf2d73784e'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:36.840638 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8d6d5d05-1578-4c53-b4c1-64cf2d73784e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e9e1d963a544a2bb59c49c067a4b797', 'request_id': 'req-8d6d5d05-1578-4c53-b4c1-64cf2d73784e'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:36.852109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8d6d5d05-1578-4c53-b4c1-64cf2d73784e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e9e1d963a544a2bb59c49c067a4b797', 'request_id': 'req-8d6d5d05-1578-4c53-b4c1-64cf2d73784e'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:36.875601 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8d6d5d05-1578-4c53-b4c1-64cf2d73784e admin admin] The token cache is being invalidate because project 5e9e1d963a544a2bb59c49c067a4b797 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:56:36.876192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8d6d5d05-1578-4c53-b4c1-64cf2d73784e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:36.879195 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-8d6d5d05-1578-4c53-b4c1-64cf2d73784e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:56:36.882170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1360/2704] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:56:36 2022] DELETE /identity/v3/projects/5e9e1d963a544a2bb59c49c067a4b797 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:36.888688 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82911e-f405-41bf-bd3e-bb73ab6d5216 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:36.889328 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82911e-f405-41bf-bd3e-bb73ab6d5216 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:36.889817 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82911e-f405-41bf-bd3e-bb73ab6d5216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:36.915730 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-ee82911e-f405-41bf-bd3e-bb73ab6d5216 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:36.917126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ee82911e-f405-41bf-bd3e-bb73ab6d5216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:37.032398 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1345/2705] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:37.039933 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f9da7-65d7-4eeb-bc76-fd5a4d511f7a None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:37.040219 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f9da7-65d7-4eeb-bc76-fd5a4d511f7a None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:37.040500 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f9da7-65d7-4eeb-bc76-fd5a4d511f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:37.066613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-e33f9da7-65d7-4eeb-bc76-fd5a4d511f7a None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:37.067865 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e33f9da7-65d7-4eeb-bc76-fd5a4d511f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:37.091534 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1361/2706] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:37.092442 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82911e-f405-41bf-bd3e-bb73ab6d5216 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:37.103772 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82911e-f405-41bf-bd3e-bb73ab6d5216 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:37.108668 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32623c-f685-46a9-bfb0-0a1a6b16b98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:37.109435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32623c-f685-46a9-bfb0-0a1a6b16b98f admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:37.109725 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32623c-f685-46a9-bfb0-0a1a6b16b98f admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:37.110116 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32623c-f685-46a9-bfb0-0a1a6b16b98f admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:37.110855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32623c-f685-46a9-bfb0-0a1a6b16b98f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:56:37.111448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32623c-f685-46a9-bfb0-0a1a6b16b98f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:37.113158 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32623c-f685-46a9-bfb0-0a1a6b16b98f admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:37.126466 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1346/2707] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:56:37.130809 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f9da7-65d7-4eeb-bc76-fd5a4d511f7a None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:37.141274 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f9da7-65d7-4eeb-bc76-fd5a4d511f7a None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:37.146182 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1bf8a9-d53a-4fb5-9700-d4bbcc2c9347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:37.146970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1bf8a9-d53a-4fb5-9700-d4bbcc2c9347 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:37.147318 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1bf8a9-d53a-4fb5-9700-d4bbcc2c9347 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:37.147773 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1bf8a9-d53a-4fb5-9700-d4bbcc2c9347 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:37.148593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1bf8a9-d53a-4fb5-9700-d4bbcc2c9347 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:37.151111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1bf8a9-d53a-4fb5-9700-d4bbcc2c9347 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:37.181823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1362/2708] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:56:37 2022] POST /identity/v3/projects => generated 409 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:56:37.186572 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32623c-f685-46a9-bfb0-0a1a6b16b98f admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:37.197042 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32623c-f685-46a9-bfb0-0a1a6b16b98f admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:37.249408 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:37.269500 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f4562-7bc5-48c2-80cd-b35b20f089aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:37.270265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f4562-7bc5-48c2-80cd-b35b20f089aa admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:37.270553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f4562-7bc5-48c2-80cd-b35b20f089aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:37.270832 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f4562-7bc5-48c2-80cd-b35b20f089aa admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:37.271632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f4562-7bc5-48c2-80cd-b35b20f089aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:37.273856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f4562-7bc5-48c2-80cd-b35b20f089aa admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:37.295087 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1347/2709] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:56:37 2022] POST /identity/v3/users => generated 305 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:56:37.299199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1bf8a9-d53a-4fb5-9700-d4bbcc2c9347 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:37.310558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1bf8a9-d53a-4fb5-9700-d4bbcc2c9347 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:37.315674 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e7a5a-b6a8-43fb-ab82-32b5e4468597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:37.316351 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e7a5a-b6a8-43fb-ab82-32b5e4468597 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:37.316685 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e7a5a-b6a8-43fb-ab82-32b5e4468597 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:37.316908 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e7a5a-b6a8-43fb-ab82-32b5e4468597 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:37.317708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e7a5a-b6a8-43fb-ab82-32b5e4468597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:37.319410 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e7a5a-b6a8-43fb-ab82-32b5e4468597 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:37.331596 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1363/2710] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:56:37.335628 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f4562-7bc5-48c2-80cd-b35b20f089aa admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:37.346246 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f4562-7bc5-48c2-80cd-b35b20f089aa admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:37.351001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77f0f5-880e-434b-a4d8-515996a69e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:37.351695 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77f0f5-880e-434b-a4d8-515996a69e16 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:37.352023 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77f0f5-880e-434b-a4d8-515996a69e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:37.352343 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77f0f5-880e-434b-a4d8-515996a69e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:37.353118 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77f0f5-880e-434b-a4d8-515996a69e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:37.354804 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e77f0f5-880e-434b-a4d8-515996a69e16 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:37.365603 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1348/2711] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:56:37.370088 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e7a5a-b6a8-43fb-ab82-32b5e4468597 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:37.381846 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e7a5a-b6a8-43fb-ab82-32b5e4468597 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:37.387954 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e57b7-6a3e-4cab-88f2-fb5cb77f55a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:37.388941 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e57b7-6a3e-4cab-88f2-fb5cb77f55a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:37.389211 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e57b7-6a3e-4cab-88f2-fb5cb77f55a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:37.389487 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e57b7-6a3e-4cab-88f2-fb5cb77f55a5 admin admin] PATH_INFO: `/v3/projects/87d2c4ded9d6402e8a09c1bf0240f530/users/72e9d61fe4dd4f34a9344f4d029612a6/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:37.408835 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e57b7-6a3e-4cab-88f2-fb5cb77f55a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d2c4ded9d6402e8a09c1bf0240f530, user_id=72e9d61fe4dd4f34a9344f4d029612a6, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:37.412354 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e57b7-6a3e-4cab-88f2-fb5cb77f55a5 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:37.424908 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:37.446299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1364/2712] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:56:37 2022] PUT /identity/v3/projects/87d2c4ded9d6402e8a09c1bf0240f530/users/72e9d61fe4dd4f34a9344f4d029612a6/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:37.457471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ead1c-b9b7-45a4-8fa7-b45658d33976 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:37.457773 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ead1c-b9b7-45a4-8fa7-b45658d33976 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:37.458061 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ead1c-b9b7-45a4-8fa7-b45658d33976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:37.486159 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-816ead1c-b9b7-45a4-8fa7-b45658d33976 None None] MFA Rules not processed for user `72e9d61fe4dd4f34a9344f4d029612a6`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:37.488132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-816ead1c-b9b7-45a4-8fa7-b45658d33976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:37.561072 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1349/2713] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:37 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:37.565641 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e57b7-6a3e-4cab-88f2-fb5cb77f55a5 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:37.639649 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e57b7-6a3e-4cab-88f2-fb5cb77f55a5 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:37.645421 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557d6ee-a106-4db3-a716-22a332cb7287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:37.646438 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557d6ee-a106-4db3-a716-22a332cb7287 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:37.646732 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557d6ee-a106-4db3-a716-22a332cb7287 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:37.647012 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557d6ee-a106-4db3-a716-22a332cb7287 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:37.647822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557d6ee-a106-4db3-a716-22a332cb7287 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:37.649734 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557d6ee-a106-4db3-a716-22a332cb7287 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:37.680587 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1365/2714] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:56:37 2022] POST /identity/v3/projects => generated 407 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:56:37.684607 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ead1c-b9b7-45a4-8fa7-b45658d33976 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:37.695179 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ead1c-b9b7-45a4-8fa7-b45658d33976 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:37.754279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aca304-9220-42d8-ae06-0241469a92cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:37.755021 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aca304-9220-42d8-ae06-0241469a92cf admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:37.755244 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aca304-9220-42d8-ae06-0241469a92cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:37.755875 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aca304-9220-42d8-ae06-0241469a92cf admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:37.756488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aca304-9220-42d8-ae06-0241469a92cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:37.758422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aca304-9220-42d8-ae06-0241469a92cf admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:37.780932 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1350/2715] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:56:37 2022] POST /identity/v3/users => generated 310 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:56:37.785180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557d6ee-a106-4db3-a716-22a332cb7287 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:37.796365 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557d6ee-a106-4db3-a716-22a332cb7287 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:37.802081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b0a02-e568-4ca3-ab8b-7f3c4effc00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:37.803056 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b0a02-e568-4ca3-ab8b-7f3c4effc00d admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:37.803342 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b0a02-e568-4ca3-ab8b-7f3c4effc00d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:37.803776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b0a02-e568-4ca3-ab8b-7f3c4effc00d admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:37.804777 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b0a02-e568-4ca3-ab8b-7f3c4effc00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:37.806758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b0a02-e568-4ca3-ab8b-7f3c4effc00d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:37.822425 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1366/2716] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 22 11:56:37.827073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aca304-9220-42d8-ae06-0241469a92cf admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:37.838264 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aca304-9220-42d8-ae06-0241469a92cf admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:37.844059 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e050d-b5f8-4fdd-a4b5-894cb58a56e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:37.845076 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e050d-b5f8-4fdd-a4b5-894cb58a56e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:37.845560 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e050d-b5f8-4fdd-a4b5-894cb58a56e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:37.845943 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e050d-b5f8-4fdd-a4b5-894cb58a56e1 admin admin] PATH_INFO: `/v3/projects/6c06f84734324f0287312683a0e687c1/users/0d7884d9d6984890a78a40993ed27378/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:37.864694 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e050d-b5f8-4fdd-a4b5-894cb58a56e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c06f84734324f0287312683a0e687c1, user_id=0d7884d9d6984890a78a40993ed27378, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:37.867611 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e050d-b5f8-4fdd-a4b5-894cb58a56e1 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:37.883095 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1351/2717] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:56:37 2022] PUT /identity/v3/projects/6c06f84734324f0287312683a0e687c1/users/0d7884d9d6984890a78a40993ed27378/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:37.887205 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b0a02-e568-4ca3-ab8b-7f3c4effc00d admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:37.898496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b0a02-e568-4ca3-ab8b-7f3c4effc00d admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:37.960250 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cf3b1-b215-450a-9b8b-04513fa3ba06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:37.960927 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cf3b1-b215-450a-9b8b-04513fa3ba06 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:37.961223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cf3b1-b215-450a-9b8b-04513fa3ba06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:37.961488 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cf3b1-b215-450a-9b8b-04513fa3ba06 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:37.962222 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85cf3b1-b215-450a-9b8b-04513fa3ba06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:37.964458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85cf3b1-b215-450a-9b8b-04513fa3ba06 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:37.976014 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1367/2718] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:56:37.980488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e050d-b5f8-4fdd-a4b5-894cb58a56e1 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:37.992635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e050d-b5f8-4fdd-a4b5-894cb58a56e1 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:37.997656 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca635b-cf25-406f-87f5-ab2441bc1d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:37.998577 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca635b-cf25-406f-87f5-ab2441bc1d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:37.998857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca635b-cf25-406f-87f5-ab2441bc1d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:37.999166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca635b-cf25-406f-87f5-ab2441bc1d0a admin admin] PATH_INFO: `/v3/projects/6c06f84734324f0287312683a0e687c1/users/0d7884d9d6984890a78a40993ed27378/roles/96c584428d924fdba9671949f760c406` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:38.001151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca635b-cf25-406f-87f5-ab2441bc1d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c06f84734324f0287312683a0e687c1, user_id=0d7884d9d6984890a78a40993ed27378, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:38.003847 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca635b-cf25-406f-87f5-ab2441bc1d0a admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:38.022116 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1352/2719] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:56:37 2022] PUT /identity/v3/projects/6c06f84734324f0287312683a0e687c1/users/0d7884d9d6984890a78a40993ed27378/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:56:38.032799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594553b3-e7a4-45e5-b19e-53e8e353df07 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:38.033337 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594553b3-e7a4-45e5-b19e-53e8e353df07 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:38.033778 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594553b3-e7a4-45e5-b19e-53e8e353df07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:38.066279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-594553b3-e7a4-45e5-b19e-53e8e353df07 None None] MFA Rules not processed for user `0d7884d9d6984890a78a40993ed27378`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:56:38.066279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-594553b3-e7a4-45e5-b19e-53e8e353df07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:38.140781 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:38.158658 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1368/2720] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:56:38 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:56:38.166487 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca635b-cf25-406f-87f5-ab2441bc1d0a admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:38.168433 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b4ca635b-cf25-406f-87f5-ab2441bc1d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:56:38.184299 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:38.248063 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca635b-cf25-406f-87f5-ab2441bc1d0a admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:38.252031 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34351189-3b75-48fd-9e3c-999f64cf5a7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:38.253459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34351189-3b75-48fd-9e3c-999f64cf5a7e service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:38.253748 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34351189-3b75-48fd-9e3c-999f64cf5a7e service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:38.254066 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34351189-3b75-48fd-9e3c-999f64cf5a7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:38.269310 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34351189-3b75-48fd-9e3c-999f64cf5a7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:38.272354 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34351189-3b75-48fd-9e3c-999f64cf5a7e service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:38.287686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1353/2721] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:56:38 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:56:38.493833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594553b3-e7a4-45e5-b19e-53e8e353df07 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:56:38.509828 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594553b3-e7a4-45e5-b19e-53e8e353df07 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:56:38.514068 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691106e0-4f3f-45f3-b7b0-7696aee4f7a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:56:38.514710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691106e0-4f3f-45f3-b7b0-7696aee4f7a3 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:56:38.515268 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691106e0-4f3f-45f3-b7b0-7696aee4f7a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:56:38.515589 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691106e0-4f3f-45f3-b7b0-7696aee4f7a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:56:38.582850 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691106e0-4f3f-45f3-b7b0-7696aee4f7a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:56:38.584385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691106e0-4f3f-45f3-b7b0-7696aee4f7a3 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:56:38.598281 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1369/2722] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:56:38 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:56:48.280345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=23 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:56:48.591119 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=30 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:57:17.856143 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34351189-3b75-48fd-9e3c-999f64cf5a7e service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:57:17.919132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34351189-3b75-48fd-9e3c-999f64cf5a7e service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:57:17.923816 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98b774-36bd-4766-9116-613d571d7fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:57:17.924685 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98b774-36bd-4766-9116-613d571d7fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:17.924969 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98b774-36bd-4766-9116-613d571d7fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:17.925248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98b774-36bd-4766-9116-613d571d7fe9 admin admin] PATH_INFO: `/v3/users/72e9d61fe4dd4f34a9344f4d029612a6` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:17.926450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98b774-36bd-4766-9116-613d571d7fe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72e9d61fe4dd4f34a9344f4d029612a6)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:57:17.928421 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98b774-36bd-4766-9116-613d571d7fe9 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:57:17.975213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cf98b774-36bd-4766-9116-613d571d7fe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72e9d61fe4dd4f34a9344f4d029612a6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:17.986933 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cf98b774-36bd-4766-9116-613d571d7fe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72e9d61fe4dd4f34a9344f4d029612a6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:17.994855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cf98b774-36bd-4766-9116-613d571d7fe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72e9d61fe4dd4f34a9344f4d029612a6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:17.996178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cf98b774-36bd-4766-9116-613d571d7fe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72e9d61fe4dd4f34a9344f4d029612a6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:18.003584 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-cf98b774-36bd-4766-9116-613d571d7fe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72e9d61fe4dd4f34a9344f4d029612a6'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:18.009545 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1354/2723] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:57:17 2022] DELETE /identity/v3/users/72e9d61fe4dd4f34a9344f4d029612a6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:57:18.014856 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e1185-f81f-4ded-bc68-4cdfdfe69ee4 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:18.015488 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e1185-f81f-4ded-bc68-4cdfdfe69ee4 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:18.015488 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e1185-f81f-4ded-bc68-4cdfdfe69ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:18.032603 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-738e1185-f81f-4ded-bc68-4cdfdfe69ee4 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:57:18.033818 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-738e1185-f81f-4ded-bc68-4cdfdfe69ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:57:18.095992 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1370/2724] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:57:18.105640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98b774-36bd-4766-9116-613d571d7fe9 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:57:18.107118 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-cf98b774-36bd-4766-9116-613d571d7fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:57:18.180222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98b774-36bd-4766-9116-613d571d7fe9 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:57:18.184733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8c5a8-60c2-4e40-9869-bdacff7d5e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:57:18.185396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8c5a8-60c2-4e40-9869-bdacff7d5e50 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:18.185715 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8c5a8-60c2-4e40-9869-bdacff7d5e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:18.186012 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8c5a8-60c2-4e40-9869-bdacff7d5e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:18.196357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8c5a8-60c2-4e40-9869-bdacff7d5e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:57:18.197627 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8c5a8-60c2-4e40-9869-bdacff7d5e50 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:57:18.208931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1355/2725] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:57:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:57:18.520047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e1185-f81f-4ded-bc68-4cdfdfe69ee4 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:57:18.521506 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-738e1185-f81f-4ded-bc68-4cdfdfe69ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:57:18.535753 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e1185-f81f-4ded-bc68-4cdfdfe69ee4 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:57:18.540599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93345b9-b866-4334-8e71-fcacf3faa65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:57:18.541524 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93345b9-b866-4334-8e71-fcacf3faa65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:18.541861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93345b9-b866-4334-8e71-fcacf3faa65a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:18.542164 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93345b9-b866-4334-8e71-fcacf3faa65a admin admin] PATH_INFO: `/v3/projects/87d2c4ded9d6402e8a09c1bf0240f530` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:18.543908 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93345b9-b866-4334-8e71-fcacf3faa65a admin admin] RBAC: Authorizing `identity:delete_project(project_id=87d2c4ded9d6402e8a09c1bf0240f530)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:57:18.545918 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93345b9-b866-4334-8e71-fcacf3faa65a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:57:18.596423 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f93345b9-b866-4334-8e71-fcacf3faa65a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87d2c4ded9d6402e8a09c1bf0240f530', 'request_id': 'req-f93345b9-b866-4334-8e71-fcacf3faa65a'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:18.602513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f93345b9-b866-4334-8e71-fcacf3faa65a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87d2c4ded9d6402e8a09c1bf0240f530', 'request_id': 'req-f93345b9-b866-4334-8e71-fcacf3faa65a'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:18.610066 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f93345b9-b866-4334-8e71-fcacf3faa65a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87d2c4ded9d6402e8a09c1bf0240f530', 'request_id': 'req-f93345b9-b866-4334-8e71-fcacf3faa65a'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:18.623217 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f93345b9-b866-4334-8e71-fcacf3faa65a admin admin] The token cache is being invalidate because project 87d2c4ded9d6402e8a09c1bf0240f530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:57:18.623536 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f93345b9-b866-4334-8e71-fcacf3faa65a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:18.624179 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f93345b9-b866-4334-8e71-fcacf3faa65a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:18.625979 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1371/2726] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:57:18 2022] DELETE /identity/v3/projects/87d2c4ded9d6402e8a09c1bf0240f530 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:57:18.630257 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8c5a8-60c2-4e40-9869-bdacff7d5e50 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:57:18.641001 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8c5a8-60c2-4e40-9869-bdacff7d5e50 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:57:18.659405 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:57:18.700582 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d336b32-e4c7-4d00-8f18-c8ec25b1a853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:57:18.701496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d336b32-e4c7-4d00-8f18-c8ec25b1a853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:18.701779 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d336b32-e4c7-4d00-8f18-c8ec25b1a853 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:18.702059 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d336b32-e4c7-4d00-8f18-c8ec25b1a853 admin admin] PATH_INFO: `/v3/users/0d7884d9d6984890a78a40993ed27378` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:18.703258 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d336b32-e4c7-4d00-8f18-c8ec25b1a853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d7884d9d6984890a78a40993ed27378)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:57:18.705262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d336b32-e4c7-4d00-8f18-c8ec25b1a853 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:57:18.754801 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-3d336b32-e4c7-4d00-8f18-c8ec25b1a853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d7884d9d6984890a78a40993ed27378'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:18.774148 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-3d336b32-e4c7-4d00-8f18-c8ec25b1a853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d7884d9d6984890a78a40993ed27378'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:18.781876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-3d336b32-e4c7-4d00-8f18-c8ec25b1a853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d7884d9d6984890a78a40993ed27378'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:18.782703 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-3d336b32-e4c7-4d00-8f18-c8ec25b1a853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d7884d9d6984890a78a40993ed27378'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:18.790360 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-3d336b32-e4c7-4d00-8f18-c8ec25b1a853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d7884d9d6984890a78a40993ed27378'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:18.798502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1356/2727] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:57:18 2022] DELETE /identity/v3/users/0d7884d9d6984890a78a40993ed27378 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:57:19.346050 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93345b9-b866-4334-8e71-fcacf3faa65a admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:57:19.348248 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f93345b9-b866-4334-8e71-fcacf3faa65a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:57:19.364221 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93345b9-b866-4334-8e71-fcacf3faa65a admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:57:19.422273 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e308c0-b6e9-4fb7-a284-a49dd66056d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:57:19.423065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e308c0-b6e9-4fb7-a284-a49dd66056d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:19.423384 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e308c0-b6e9-4fb7-a284-a49dd66056d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:19.423674 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e308c0-b6e9-4fb7-a284-a49dd66056d9 admin admin] PATH_INFO: `/v3/projects/6c06f84734324f0287312683a0e687c1` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:19.424880 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e308c0-b6e9-4fb7-a284-a49dd66056d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c06f84734324f0287312683a0e687c1)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:57:19.426812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e308c0-b6e9-4fb7-a284-a49dd66056d9 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:57:19.462749 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:57:19.483392 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-65e308c0-b6e9-4fb7-a284-a49dd66056d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c06f84734324f0287312683a0e687c1', 'request_id': 'req-65e308c0-b6e9-4fb7-a284-a49dd66056d9'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:19.490379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-65e308c0-b6e9-4fb7-a284-a49dd66056d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c06f84734324f0287312683a0e687c1', 'request_id': 'req-65e308c0-b6e9-4fb7-a284-a49dd66056d9'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:19.500594 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-65e308c0-b6e9-4fb7-a284-a49dd66056d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c06f84734324f0287312683a0e687c1', 'request_id': 'req-65e308c0-b6e9-4fb7-a284-a49dd66056d9'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:19.512747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-65e308c0-b6e9-4fb7-a284-a49dd66056d9 admin admin] The token cache is being invalidate because project 6c06f84734324f0287312683a0e687c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:57:19.513106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-65e308c0-b6e9-4fb7-a284-a49dd66056d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:19.514034 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-65e308c0-b6e9-4fb7-a284-a49dd66056d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:19.516416 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1372/2728] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:57:19 2022] DELETE /identity/v3/projects/6c06f84734324f0287312683a0e687c1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:57:19.522396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0fdba-c379-4b60-84b8-fe3c14b93cd7 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:19.522946 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0fdba-c379-4b60-84b8-fe3c14b93cd7 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:19.523043 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0fdba-c379-4b60-84b8-fe3c14b93cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:19.543165 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-6bb0fdba-c379-4b60-84b8-fe3c14b93cd7 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:57:19.544498 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-6bb0fdba-c379-4b60-84b8-fe3c14b93cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:57:19.615613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1357/2729] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:57:19.624594 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1704ea0-a850-4f61-bb9b-67d07a60ab5d None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:19.624922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1704ea0-a850-4f61-bb9b-67d07a60ab5d None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:19.625174 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1704ea0-a850-4f61-bb9b-67d07a60ab5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:19.645052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-d1704ea0-a850-4f61-bb9b-67d07a60ab5d None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:57:19.646274 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d1704ea0-a850-4f61-bb9b-67d07a60ab5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:57:19.655007 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1373/2730] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:57:19.659380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0fdba-c379-4b60-84b8-fe3c14b93cd7 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:57:19.669999 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0fdba-c379-4b60-84b8-fe3c14b93cd7 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:57:19.674877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed21718-7a2a-427d-b01e-1c59534cb3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:57:19.675583 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed21718-7a2a-427d-b01e-1c59534cb3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:19.675873 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed21718-7a2a-427d-b01e-1c59534cb3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:19.676151 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed21718-7a2a-427d-b01e-1c59534cb3a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:19.676878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed21718-7a2a-427d-b01e-1c59534cb3a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:57:19.677189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed21718-7a2a-427d-b01e-1c59534cb3a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:57:19.678785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed21718-7a2a-427d-b01e-1c59534cb3a4 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:57:19.690719 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1358/2731] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:57:19.695159 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1704ea0-a850-4f61-bb9b-67d07a60ab5d None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:57:19.706487 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1704ea0-a850-4f61-bb9b-67d07a60ab5d None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:57:19.712408 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e9769-183b-45d6-8b32-c7f2dd64cfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:57:19.713082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e9769-183b-45d6-8b32-c7f2dd64cfe5 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:19.713418 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e9769-183b-45d6-8b32-c7f2dd64cfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:19.713693 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e9769-183b-45d6-8b32-c7f2dd64cfe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:19.715006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e9769-183b-45d6-8b32-c7f2dd64cfe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:57:19.717129 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e9769-183b-45d6-8b32-c7f2dd64cfe5 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:57:19.742755 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1374/2732] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:57:19 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:57:19.747045 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed21718-7a2a-427d-b01e-1c59534cb3a4 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:57:19.757255 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed21718-7a2a-427d-b01e-1c59534cb3a4 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:57:19.785100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:57:19.817355 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f201a0-d346-47f3-b574-29ad8ab8c324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:57:19.818026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f201a0-d346-47f3-b574-29ad8ab8c324 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:19.818316 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f201a0-d346-47f3-b574-29ad8ab8c324 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:19.818594 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f201a0-d346-47f3-b574-29ad8ab8c324 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:19.819291 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f201a0-d346-47f3-b574-29ad8ab8c324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:57:19.821218 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f201a0-d346-47f3-b574-29ad8ab8c324 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:57:19.845394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1359/2733] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:57:19 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:57:19.849528 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e9769-183b-45d6-8b32-c7f2dd64cfe5 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:57:19.859848 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e9769-183b-45d6-8b32-c7f2dd64cfe5 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:57:19.865190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8905b-c23c-4d93-9816-bb704fab2f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:57:19.865880 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8905b-c23c-4d93-9816-bb704fab2f8d admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:19.866166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8905b-c23c-4d93-9816-bb704fab2f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:19.866444 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8905b-c23c-4d93-9816-bb704fab2f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:19.867169 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a8905b-c23c-4d93-9816-bb704fab2f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:57:19.868829 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a8905b-c23c-4d93-9816-bb704fab2f8d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:57:19.879452 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1375/2734] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:57:19.883612 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f201a0-d346-47f3-b574-29ad8ab8c324 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:57:19.896041 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f201a0-d346-47f3-b574-29ad8ab8c324 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:57:19.901024 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91156061-0542-4e29-a6a6-cd6cf090d743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:57:19.902610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91156061-0542-4e29-a6a6-cd6cf090d743 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:19.903322 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91156061-0542-4e29-a6a6-cd6cf090d743 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:19.903821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91156061-0542-4e29-a6a6-cd6cf090d743 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:19.905051 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91156061-0542-4e29-a6a6-cd6cf090d743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:57:19.907893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91156061-0542-4e29-a6a6-cd6cf090d743 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:57:19.925133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1360/2735] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:57:19.929416 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8905b-c23c-4d93-9816-bb704fab2f8d admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:57:19.942159 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8905b-c23c-4d93-9816-bb704fab2f8d admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:57:19.947942 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83729628-e4a9-4d02-83a1-dd29f78fa555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:57:19.949416 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83729628-e4a9-4d02-83a1-dd29f78fa555 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:19.949841 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83729628-e4a9-4d02-83a1-dd29f78fa555 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:19.950153 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83729628-e4a9-4d02-83a1-dd29f78fa555 admin admin] PATH_INFO: `/v3/projects/3d843ffec4d24257a51a1fd4ea4cdcd2/users/e10274cc15a645b092eef7e9723ea277/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:19.967141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83729628-e4a9-4d02-83a1-dd29f78fa555 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d843ffec4d24257a51a1fd4ea4cdcd2, user_id=e10274cc15a645b092eef7e9723ea277, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:57:19.969815 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83729628-e4a9-4d02-83a1-dd29f78fa555 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:57:19.983484 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1376/2736] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:57:19 2022] PUT /identity/v3/projects/3d843ffec4d24257a51a1fd4ea4cdcd2/users/e10274cc15a645b092eef7e9723ea277/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:57:19.993865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e82cb-e8af-48b9-8632-fc5c7c7dba29 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:19.994161 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e82cb-e8af-48b9-8632-fc5c7c7dba29 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:19.994449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e82cb-e8af-48b9-8632-fc5c7c7dba29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:20.022729 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-492e82cb-e8af-48b9-8632-fc5c7c7dba29 None None] MFA Rules not processed for user `e10274cc15a645b092eef7e9723ea277`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:57:20.024015 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-492e82cb-e8af-48b9-8632-fc5c7c7dba29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:57:20.090963 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1361/2737] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:57:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:57:20.102538 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83729628-e4a9-4d02-83a1-dd29f78fa555 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:57:20.104231 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-83729628-e4a9-4d02-83a1-dd29f78fa555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:57:20.163222 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83729628-e4a9-4d02-83a1-dd29f78fa555 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:57:20.166970 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889c147-dcd5-4649-bc71-ac23410c2b15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:57:20.168212 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889c147-dcd5-4649-bc71-ac23410c2b15 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:20.168500 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889c147-dcd5-4649-bc71-ac23410c2b15 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:20.168784 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889c147-dcd5-4649-bc71-ac23410c2b15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:20.177684 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889c147-dcd5-4649-bc71-ac23410c2b15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:57:20.179060 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889c147-dcd5-4649-bc71-ac23410c2b15 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:57:20.188412 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1377/2738] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:57:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:57:22.203481 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c86be-9a4c-487a-91d3-8b5c187a677a None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:22.203966 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c86be-9a4c-487a-91d3-8b5c187a677a None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:22.204074 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c86be-9a4c-487a-91d3-8b5c187a677a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:22.222507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-1f8c86be-9a4c-487a-91d3-8b5c187a677a None None] MFA Rules not processed for user `955c44ffe1bd426e900c7405df44d1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:57:22.223783 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-1f8c86be-9a4c-487a-91d3-8b5c187a677a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:57:22.280130 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1362/2739] 10.222.0.36 () {58 vars in 967 bytes} [Wed Jun 22 11:57:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:57:22.301772 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889c147-dcd5-4649-bc71-ac23410c2b15 service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:57:22.303185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b889c147-dcd5-4649-bc71-ac23410c2b15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:57:22.356974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:57:22.374741 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889c147-dcd5-4649-bc71-ac23410c2b15 service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:57:22.379734 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd14f8-f228-4e38-8f32-917e5c513e4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dcd642e290e44e67b56eb3b8361d7af3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:57:22.380379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd14f8-f228-4e38-8f32-917e5c513e4c service swift] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:22.380661 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd14f8-f228-4e38-8f32-917e5c513e4c service swift] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:22.380936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd14f8-f228-4e38-8f32-917e5c513e4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:22.389252 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd14f8-f228-4e38-8f32-917e5c513e4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:57:22.390853 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd14f8-f228-4e38-8f32-917e5c513e4c service swift] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:57:22.399068 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1378/2740] 10.222.0.36 () {60 vars in 1384 bytes} [Wed Jun 22 11:57:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 22 11:57:32.273394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:57:32.395941 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=8 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:57:42.925862 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c86be-9a4c-487a-91d3-8b5c187a677a None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:57:43.044306 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c86be-9a4c-487a-91d3-8b5c187a677a None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:57:43.049225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdadef2-2afd-4996-a5f0-3191083bc4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:57:43.050188 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdadef2-2afd-4996-a5f0-3191083bc4f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:43.050377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdadef2-2afd-4996-a5f0-3191083bc4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:43.050657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdadef2-2afd-4996-a5f0-3191083bc4f3 admin admin] PATH_INFO: `/v3/users/e10274cc15a645b092eef7e9723ea277` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:43.051922 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdadef2-2afd-4996-a5f0-3191083bc4f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e10274cc15a645b092eef7e9723ea277)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:57:43.053876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdadef2-2afd-4996-a5f0-3191083bc4f3 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:57:43.099809 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8fdadef2-2afd-4996-a5f0-3191083bc4f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e10274cc15a645b092eef7e9723ea277'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:43.112018 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8fdadef2-2afd-4996-a5f0-3191083bc4f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e10274cc15a645b092eef7e9723ea277'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:43.121032 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8fdadef2-2afd-4996-a5f0-3191083bc4f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e10274cc15a645b092eef7e9723ea277'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:43.121670 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8fdadef2-2afd-4996-a5f0-3191083bc4f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e10274cc15a645b092eef7e9723ea277'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:43.128769 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-8fdadef2-2afd-4996-a5f0-3191083bc4f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e10274cc15a645b092eef7e9723ea277'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:43.135210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1363/2741] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:57:42 2022] DELETE /identity/v3/users/e10274cc15a645b092eef7e9723ea277 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:57:43.139735 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dd4ed-8bc7-4a5e-b61d-41fe39abb7b8 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:43.140239 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dd4ed-8bc7-4a5e-b61d-41fe39abb7b8 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:43.140379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dd4ed-8bc7-4a5e-b61d-41fe39abb7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:43.189810 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-564dd4ed-8bc7-4a5e-b61d-41fe39abb7b8 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:57:43.191022 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-564dd4ed-8bc7-4a5e-b61d-41fe39abb7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:57:43.251062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1379/2742] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:57:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:57:43.261235 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdadef2-2afd-4996-a5f0-3191083bc4f3 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:57:43.262736 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-8fdadef2-2afd-4996-a5f0-3191083bc4f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:57:43.308524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:57:43.363944 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdadef2-2afd-4996-a5f0-3191083bc4f3 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:57:43.367724 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df2201-9a94-40ed-815d-49bd648b02fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:57:43.368370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df2201-9a94-40ed-815d-49bd648b02fa service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:43.368658 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df2201-9a94-40ed-815d-49bd648b02fa service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:43.368932 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df2201-9a94-40ed-815d-49bd648b02fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:43.379304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df2201-9a94-40ed-815d-49bd648b02fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:57:43.380737 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df2201-9a94-40ed-815d-49bd648b02fa service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:57:43.390849 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1364/2743] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:57:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:57:43.664519 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564dd4ed-8bc7-4a5e-b61d-41fe39abb7b8 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:57:43.666047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-564dd4ed-8bc7-4a5e-b61d-41fe39abb7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:57:43.678471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564dd4ed-8bc7-4a5e-b61d-41fe39abb7b8 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:57:43.684065 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33e648-2b3f-42c9-a812-3523b3e76d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:57:43.685006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33e648-2b3f-42c9-a812-3523b3e76d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:43.685377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33e648-2b3f-42c9-a812-3523b3e76d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:43.685798 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33e648-2b3f-42c9-a812-3523b3e76d26 admin admin] PATH_INFO: `/v3/projects/3d843ffec4d24257a51a1fd4ea4cdcd2` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:43.687068 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa33e648-2b3f-42c9-a812-3523b3e76d26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d843ffec4d24257a51a1fd4ea4cdcd2)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:57:43.689184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa33e648-2b3f-42c9-a812-3523b3e76d26 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:57:43.736743 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-fa33e648-2b3f-42c9-a812-3523b3e76d26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d843ffec4d24257a51a1fd4ea4cdcd2', 'request_id': 'req-fa33e648-2b3f-42c9-a812-3523b3e76d26'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:43.743149 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-fa33e648-2b3f-42c9-a812-3523b3e76d26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d843ffec4d24257a51a1fd4ea4cdcd2', 'request_id': 'req-fa33e648-2b3f-42c9-a812-3523b3e76d26'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:43.750879 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-fa33e648-2b3f-42c9-a812-3523b3e76d26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d843ffec4d24257a51a1fd4ea4cdcd2', 'request_id': 'req-fa33e648-2b3f-42c9-a812-3523b3e76d26'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:43.761187 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-fa33e648-2b3f-42c9-a812-3523b3e76d26 admin admin] The token cache is being invalidate because project 3d843ffec4d24257a51a1fd4ea4cdcd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:57:43.761471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-fa33e648-2b3f-42c9-a812-3523b3e76d26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:43.762760 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-fa33e648-2b3f-42c9-a812-3523b3e76d26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:57:43.764339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1380/2744] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:57:43 2022] DELETE /identity/v3/projects/3d843ffec4d24257a51a1fd4ea4cdcd2 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 11:57:53.386008 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=25 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:57:53.758281 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=66,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:57:58.291493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae385b9-75e6-4427-99ac-22fce9045152 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:58.291981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae385b9-75e6-4427-99ac-22fce9045152 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:58.292117 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae385b9-75e6-4427-99ac-22fce9045152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:58.310369 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-8ae385b9-75e6-4427-99ac-22fce9045152 None None] MFA Rules not processed for user `239de565f79f474bb21699fd9adc96c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:57:58.311650 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-8ae385b9-75e6-4427-99ac-22fce9045152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:57:58.370288 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1365/2745] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:57:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:57:58.709729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33e648-2b3f-42c9-a812-3523b3e76d26 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:57:58.711225 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-fa33e648-2b3f-42c9-a812-3523b3e76d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:57:58.769425 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33e648-2b3f-42c9-a812-3523b3e76d26 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:57:58.773240 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a12ec-57d4-40c2-b8bd-9f5db0240ca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:57:58.773898 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a12ec-57d4-40c2-b8bd-9f5db0240ca8 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:57:58.774182 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a12ec-57d4-40c2-b8bd-9f5db0240ca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:57:58.774460 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a12ec-57d4-40c2-b8bd-9f5db0240ca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:57:58.776325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b76a12ec-57d4-40c2-b8bd-9f5db0240ca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:57:58.786975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a12ec-57d4-40c2-b8bd-9f5db0240ca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:57:58.788467 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76a12ec-57d4-40c2-b8bd-9f5db0240ca8 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:57:58.797214 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1381/2746] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:57:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:58:08.363367 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=25 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:58:08.792901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=26 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:58:17.386166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae385b9-75e6-4427-99ac-22fce9045152 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:58:17.387991 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-8ae385b9-75e6-4427-99ac-22fce9045152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:17.470861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae385b9-75e6-4427-99ac-22fce9045152 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:58:17.476377 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc52ee-20e2-4284-ac91-d6f320a48310 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:58:17.477038 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc52ee-20e2-4284-ac91-d6f320a48310 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:17.477323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc52ee-20e2-4284-ac91-d6f320a48310 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:17.477603 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc52ee-20e2-4284-ac91-d6f320a48310 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:17.479417 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-acdc52ee-20e2-4284-ac91-d6f320a48310 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:17.494175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc52ee-20e2-4284-ac91-d6f320a48310 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:58:17.495812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc52ee-20e2-4284-ac91-d6f320a48310 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:58:17.507051 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1366/2747] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:58:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:58:21.539330 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a12ec-57d4-40c2-b8bd-9f5db0240ca8 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:58:21.540779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b76a12ec-57d4-40c2-b8bd-9f5db0240ca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:21.630936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a12ec-57d4-40c2-b8bd-9f5db0240ca8 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:58:21.635938 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535adf5e-bd5c-4477-9945-f4fe9aaeb86c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93630ed5e6dc4a5f958bc7cf1a68a26c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:58:21.636569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535adf5e-bd5c-4477-9945-f4fe9aaeb86c service placement] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:21.636851 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535adf5e-bd5c-4477-9945-f4fe9aaeb86c service placement] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:21.637130 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535adf5e-bd5c-4477-9945-f4fe9aaeb86c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:21.638857 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-535adf5e-bd5c-4477-9945-f4fe9aaeb86c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:21.650208 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535adf5e-bd5c-4477-9945-f4fe9aaeb86c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:58:21.651810 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535adf5e-bd5c-4477-9945-f4fe9aaeb86c service placement] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:58:21.661571 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1382/2748] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:58:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:58:22.383838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc52ee-20e2-4284-ac91-d6f320a48310 service nova] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:58:22.385239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-acdc52ee-20e2-4284-ac91-d6f320a48310 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:22.452331 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc52ee-20e2-4284-ac91-d6f320a48310 service nova] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:58:22.456028 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64c622-17ea-4fb3-8d37-6de76bbf3c8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:58:22.456721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64c622-17ea-4fb3-8d37-6de76bbf3c8d service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:22.457006 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64c622-17ea-4fb3-8d37-6de76bbf3c8d service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:22.457283 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64c622-17ea-4fb3-8d37-6de76bbf3c8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:22.459079 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ac64c622-17ea-4fb3-8d37-6de76bbf3c8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:22.545345 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64c622-17ea-4fb3-8d37-6de76bbf3c8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:58:22.546714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64c622-17ea-4fb3-8d37-6de76bbf3c8d service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:58:22.565619 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1367/2749] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:58:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:58:25.392777 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535adf5e-bd5c-4477-9945-f4fe9aaeb86c service placement] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:58:25.410031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535adf5e-bd5c-4477-9945-f4fe9aaeb86c service placement] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:58:25.416510 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73eb182-2967-4f7e-bfac-c0dc01c0db09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:58:25.417167 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73eb182-2967-4f7e-bfac-c0dc01c0db09 service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:25.417454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73eb182-2967-4f7e-bfac-c0dc01c0db09 service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:25.417733 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73eb182-2967-4f7e-bfac-c0dc01c0db09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:25.419590 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d73eb182-2967-4f7e-bfac-c0dc01c0db09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:25.436204 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73eb182-2967-4f7e-bfac-c0dc01c0db09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:58:25.437849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73eb182-2967-4f7e-bfac-c0dc01c0db09 service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:58:25.455908 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1383/2750] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:58:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 11:58:29.360166 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64c622-17ea-4fb3-8d37-6de76bbf3c8d service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:58:29.361602 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ac64c622-17ea-4fb3-8d37-6de76bbf3c8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:29.369448 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:58:29.374294 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64c622-17ea-4fb3-8d37-6de76bbf3c8d service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:58:29.378309 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4967ce-47cc-416f-aa1b-4c1352f1e4c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:58:29.378946 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4967ce-47cc-416f-aa1b-4c1352f1e4c3 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:29.379228 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4967ce-47cc-416f-aa1b-4c1352f1e4c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:29.379534 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4967ce-47cc-416f-aa1b-4c1352f1e4c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:29.381273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-aa4967ce-47cc-416f-aa1b-4c1352f1e4c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:29.456848 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4967ce-47cc-416f-aa1b-4c1352f1e4c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:58:29.458290 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4967ce-47cc-416f-aa1b-4c1352f1e4c3 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:58:29.469008 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1368/2751] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:58:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:58:32.561993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73eb182-2967-4f7e-bfac-c0dc01c0db09 service nova] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:58:32.566410 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-d73eb182-2967-4f7e-bfac-c0dc01c0db09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:32.583157 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73eb182-2967-4f7e-bfac-c0dc01c0db09 service nova] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:58:32.588047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969bcf0-77b6-4539-b340-cdefecb4cb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:58:32.588861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969bcf0-77b6-4539-b340-cdefecb4cb70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:32.589199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969bcf0-77b6-4539-b340-cdefecb4cb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:32.589408 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969bcf0-77b6-4539-b340-cdefecb4cb70 admin admin] PATH_INFO: `/v3/users/034858240ac34e549aa5e0754726f9c6` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:32.590647 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969bcf0-77b6-4539-b340-cdefecb4cb70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=034858240ac34e549aa5e0754726f9c6)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:58:32.592708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969bcf0-77b6-4539-b340-cdefecb4cb70 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:58:32.644561 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-4969bcf0-77b6-4539-b340-cdefecb4cb70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '034858240ac34e549aa5e0754726f9c6'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:58:32.657180 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-4969bcf0-77b6-4539-b340-cdefecb4cb70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '034858240ac34e549aa5e0754726f9c6'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:58:32.669064 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-4969bcf0-77b6-4539-b340-cdefecb4cb70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '034858240ac34e549aa5e0754726f9c6'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:58:32.676098 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-4969bcf0-77b6-4539-b340-cdefecb4cb70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '034858240ac34e549aa5e0754726f9c6'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:58:32.676847 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-4969bcf0-77b6-4539-b340-cdefecb4cb70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '034858240ac34e549aa5e0754726f9c6'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:58:32.682678 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1384/2752] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 11:58:32 2022] DELETE /identity/v3/users/034858240ac34e549aa5e0754726f9c6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:58:32.854968 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4967ce-47cc-416f-aa1b-4c1352f1e4c3 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:58:32.868396 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4967ce-47cc-416f-aa1b-4c1352f1e4c3 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:58:32.928733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0e092-2d0f-449a-b234-c56fb8335915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:58:32.929525 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0e092-2d0f-449a-b234-c56fb8335915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:32.929812 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0e092-2d0f-449a-b234-c56fb8335915 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:32.930095 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0e092-2d0f-449a-b234-c56fb8335915 admin admin] PATH_INFO: `/v3/projects/6f64ad44d3c247caa7d106861916f245` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:32.931305 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b0e092-2d0f-449a-b234-c56fb8335915 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f64ad44d3c247caa7d106861916f245)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:58:32.933248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b0e092-2d0f-449a-b234-c56fb8335915 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:58:32.979178 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-33b0e092-2d0f-449a-b234-c56fb8335915 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f64ad44d3c247caa7d106861916f245', 'request_id': 'req-33b0e092-2d0f-449a-b234-c56fb8335915'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:58:32.986605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-33b0e092-2d0f-449a-b234-c56fb8335915 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f64ad44d3c247caa7d106861916f245', 'request_id': 'req-33b0e092-2d0f-449a-b234-c56fb8335915'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:58:32.993253 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-33b0e092-2d0f-449a-b234-c56fb8335915 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f64ad44d3c247caa7d106861916f245', 'request_id': 'req-33b0e092-2d0f-449a-b234-c56fb8335915'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:58:33.003777 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-33b0e092-2d0f-449a-b234-c56fb8335915 admin admin] The token cache is being invalidate because project 6f64ad44d3c247caa7d106861916f245 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 11:58:33.004063 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-33b0e092-2d0f-449a-b234-c56fb8335915 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:58:33.004717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-33b0e092-2d0f-449a-b234-c56fb8335915 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 11:58:33.006287 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1369/2753] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 11:58:32 2022] DELETE /identity/v3/projects/6f64ad44d3c247caa7d106861916f245 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:58:33.011196 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabe278-7cc4-4623-b75a-76d685565b5a None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:33.011528 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabe278-7cc4-4623-b75a-76d685565b5a None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:33.011807 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabe278-7cc4-4623-b75a-76d685565b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:33.049403 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:58:33.059948 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-ffabe278-7cc4-4623-b75a-76d685565b5a None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:58:33.061160 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ffabe278-7cc4-4623-b75a-76d685565b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:33.119434 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1385/2754] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:58:33.127094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51883e-8008-4192-824a-db7665304a16 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:33.127412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51883e-8008-4192-824a-db7665304a16 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:33.127694 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51883e-8008-4192-824a-db7665304a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:33.146825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-2f51883e-8008-4192-824a-db7665304a16 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:58:33.148069 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2f51883e-8008-4192-824a-db7665304a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:33.156613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1370/2755] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:58:33.160785 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabe278-7cc4-4623-b75a-76d685565b5a None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:58:33.170838 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabe278-7cc4-4623-b75a-76d685565b5a None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:58:33.175626 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2919b9-c2b1-45b1-90d2-83b217c422b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:58:33.176280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2919b9-c2b1-45b1-90d2-83b217c422b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:33.176566 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2919b9-c2b1-45b1-90d2-83b217c422b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:33.176901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2919b9-c2b1-45b1-90d2-83b217c422b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:33.177570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2919b9-c2b1-45b1-90d2-83b217c422b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 11:58:33.177878 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2919b9-c2b1-45b1-90d2-83b217c422b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:58:33.179521 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2919b9-c2b1-45b1-90d2-83b217c422b1 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:58:33.191955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1386/2756] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 11:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 11:58:33.196283 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51883e-8008-4192-824a-db7665304a16 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:58:33.206128 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51883e-8008-4192-824a-db7665304a16 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:58:33.211007 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf83ab-e109-480f-aafd-8dac792f3eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:58:33.211703 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf83ab-e109-480f-aafd-8dac792f3eeb admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:33.211987 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf83ab-e109-480f-aafd-8dac792f3eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:33.212263 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf83ab-e109-480f-aafd-8dac792f3eeb admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:33.213940 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bf83ab-e109-480f-aafd-8dac792f3eeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:58:33.215891 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bf83ab-e109-480f-aafd-8dac792f3eeb admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:58:33.235123 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:58:33.245471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1371/2757] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 11:58:33 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 11:58:33.246448 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2919b9-c2b1-45b1-90d2-83b217c422b1 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:58:33.257022 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2919b9-c2b1-45b1-90d2-83b217c422b1 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:58:33.313390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2415f80-0b7f-4e06-8caa-139bd5386e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:58:33.314058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2415f80-0b7f-4e06-8caa-139bd5386e9a admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:33.314371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2415f80-0b7f-4e06-8caa-139bd5386e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:33.314647 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2415f80-0b7f-4e06-8caa-139bd5386e9a admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:33.315339 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2415f80-0b7f-4e06-8caa-139bd5386e9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:58:33.317351 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2415f80-0b7f-4e06-8caa-139bd5386e9a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:58:33.340840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1387/2758] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 11:58:33 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 11:58:33.344815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf83ab-e109-480f-aafd-8dac792f3eeb admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:58:33.355510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf83ab-e109-480f-aafd-8dac792f3eeb admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:58:33.360414 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb96ac7-795c-4842-ba74-42edc0cc8d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:58:33.361098 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb96ac7-795c-4842-ba74-42edc0cc8d3e admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:33.361385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb96ac7-795c-4842-ba74-42edc0cc8d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:33.361660 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb96ac7-795c-4842-ba74-42edc0cc8d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:33.362380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb96ac7-795c-4842-ba74-42edc0cc8d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:58:33.364005 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb96ac7-795c-4842-ba74-42edc0cc8d3e admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:58:33.375256 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1372/2759] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:58:33.379643 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2415f80-0b7f-4e06-8caa-139bd5386e9a admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:58:33.390714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2415f80-0b7f-4e06-8caa-139bd5386e9a admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:58:33.396326 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba07f6-be7a-4327-b560-492bacd1eb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:58:33.397038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba07f6-be7a-4327-b560-492bacd1eb26 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:33.397302 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba07f6-be7a-4327-b560-492bacd1eb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:33.397950 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba07f6-be7a-4327-b560-492bacd1eb26 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:33.398315 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba07f6-be7a-4327-b560-492bacd1eb26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:58:33.400013 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba07f6-be7a-4327-b560-492bacd1eb26 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:58:33.411035 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1388/2760] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 11:58:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 11:58:33.415104 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb96ac7-795c-4842-ba74-42edc0cc8d3e admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:58:33.425480 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb96ac7-795c-4842-ba74-42edc0cc8d3e admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:58:33.430328 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f57023-99b8-4d4c-be1d-bd812e3bb2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:58:33.431225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f57023-99b8-4d4c-be1d-bd812e3bb2d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:33.431587 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f57023-99b8-4d4c-be1d-bd812e3bb2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:33.431869 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f57023-99b8-4d4c-be1d-bd812e3bb2d7 admin admin] PATH_INFO: `/v3/projects/897ec8bf954c41648da9cac00bc44749/users/f2d34a97831f4a7c885ebe64aecb3669/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:33.448340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f57023-99b8-4d4c-be1d-bd812e3bb2d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=897ec8bf954c41648da9cac00bc44749, user_id=f2d34a97831f4a7c885ebe64aecb3669, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:58:33.451941 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f57023-99b8-4d4c-be1d-bd812e3bb2d7 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:58:33.466784 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1373/2761] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 11:58:33 2022] PUT /identity/v3/projects/897ec8bf954c41648da9cac00bc44749/users/f2d34a97831f4a7c885ebe64aecb3669/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 11:58:33.472160 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d196a7-d349-4b3c-a9ec-72ecaef065ee None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:33.472471 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d196a7-d349-4b3c-a9ec-72ecaef065ee None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:33.472740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d196a7-d349-4b3c-a9ec-72ecaef065ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:33.490800 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f7d196a7-d349-4b3c-a9ec-72ecaef065ee None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:58:33.492073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f7d196a7-d349-4b3c-a9ec-72ecaef065ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:33.567425 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1389/2762] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 11:58:33.577140 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f57023-99b8-4d4c-be1d-bd812e3bb2d7 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:58:33.578951 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b2f57023-99b8-4d4c-be1d-bd812e3bb2d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:33.636612 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f57023-99b8-4d4c-be1d-bd812e3bb2d7 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:58:33.640246 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18833c92-008b-46cf-b458-41fd9622b8b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:58:33.640900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18833c92-008b-46cf-b458-41fd9622b8b3 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:33.641185 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18833c92-008b-46cf-b458-41fd9622b8b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:33.641464 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18833c92-008b-46cf-b458-41fd9622b8b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:33.650667 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18833c92-008b-46cf-b458-41fd9622b8b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:58:33.652135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18833c92-008b-46cf-b458-41fd9622b8b3 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:58:33.661510 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1374/2763] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:58:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:58:37.593954 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408c07b-b30a-488f-bb6c-d9c851bb9e6c None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:37.594329 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408c07b-b30a-488f-bb6c-d9c851bb9e6c None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:37.594442 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408c07b-b30a-488f-bb6c-d9c851bb9e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:37.615761 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:58:37.626030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f408c07b-b30a-488f-bb6c-d9c851bb9e6c None None] MFA Rules not processed for user `f2d34a97831f4a7c885ebe64aecb3669`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:58:37.627244 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f408c07b-b30a-488f-bb6c-d9c851bb9e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:37.717943 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1390/2764] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:58:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:58:37.733708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18833c92-008b-46cf-b458-41fd9622b8b3 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:58:37.735107 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-18833c92-008b-46cf-b458-41fd9622b8b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:37.831227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18833c92-008b-46cf-b458-41fd9622b8b3 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:58:37.837043 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1f4a0-79ca-4a0a-bab8-02b0e4e27d04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:58:37.837789 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1f4a0-79ca-4a0a-bab8-02b0e4e27d04 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:37.838149 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1f4a0-79ca-4a0a-bab8-02b0e4e27d04 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:37.838531 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1f4a0-79ca-4a0a-bab8-02b0e4e27d04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:37.848216 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1f4a0-79ca-4a0a-bab8-02b0e4e27d04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:58:37.849857 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c1f4a0-79ca-4a0a-bab8-02b0e4e27d04 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:58:37.861127 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1375/2765] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:58:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:58:40.267480 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534721f-f9ca-434a-9f8d-1bf7a12c49a1 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:40.267947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534721f-f9ca-434a-9f8d-1bf7a12c49a1 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:40.268032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534721f-f9ca-434a-9f8d-1bf7a12c49a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:40.288811 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-3534721f-f9ca-434a-9f8d-1bf7a12c49a1 None None] MFA Rules not processed for user `f2d34a97831f4a7c885ebe64aecb3669`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 11:58:40.290129 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-3534721f-f9ca-434a-9f8d-1bf7a12c49a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:40.306197 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1391/2766] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 11:58:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 11:58:40.667095 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1f4a0-79ca-4a0a-bab8-02b0e4e27d04 service nova] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 11:58:40.668658 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c2c1f4a0-79ca-4a0a-bab8-02b0e4e27d04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:40.687463 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c1f4a0-79ca-4a0a-bab8-02b0e4e27d04 service nova] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 11:58:40.694008 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352444b7-43af-4ab5-80c3-d5580188b95f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 11:58:40.694799 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352444b7-43af-4ab5-80c3-d5580188b95f service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 11:58:40.695324 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352444b7-43af-4ab5-80c3-d5580188b95f service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 11:58:40.695699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352444b7-43af-4ab5-80c3-d5580188b95f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 11:58:40.697634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-352444b7-43af-4ab5-80c3-d5580188b95f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 11:58:40.708902 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352444b7-43af-4ab5-80c3-d5580188b95f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 11:58:40.710342 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352444b7-43af-4ab5-80c3-d5580188b95f service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 11:58:40.719795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1376/2767] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 11:58:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 11:58:50.284678 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=30 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 11:58:50.715100 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=81,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:00:20.104527 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534721f-f9ca-434a-9f8d-1bf7a12c49a1 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:00:20.106058 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-3534721f-f9ca-434a-9f8d-1bf7a12c49a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:00:20.162522 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534721f-f9ca-434a-9f8d-1bf7a12c49a1 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:00:20.166112 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb3b12-481a-4a07-b454-09323d45248f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:00:20.166760 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb3b12-481a-4a07-b454-09323d45248f service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:00:20.167050 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb3b12-481a-4a07-b454-09323d45248f service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:00:20.167329 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb3b12-481a-4a07-b454-09323d45248f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:00:20.169125 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-1efb3b12-481a-4a07-b454-09323d45248f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:00:20.226606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb3b12-481a-4a07-b454-09323d45248f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:00:20.228014 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb3b12-481a-4a07-b454-09323d45248f service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:00:20.237172 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1392/2768] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:00:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:00:30.232796 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=48 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:00:32.262032 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e15b54-b936-417f-91e3-e83387586a4b None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:00:32.262534 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e15b54-b936-417f-91e3-e83387586a4b None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:00:32.262635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e15b54-b936-417f-91e3-e83387586a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:00:32.280540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-54e15b54-b936-417f-91e3-e83387586a4b None None] MFA Rules not processed for user `239de565f79f474bb21699fd9adc96c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:00:32.281759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-54e15b54-b936-417f-91e3-e83387586a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:00:32.289176 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1377/2769] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:00:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:00:35.243608 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb3b12-481a-4a07-b454-09323d45248f service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:00:35.255263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb3b12-481a-4a07-b454-09323d45248f service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:00:35.259467 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6af476-e71d-4a82-af95-75fdc8a58b78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:00:35.260200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6af476-e71d-4a82-af95-75fdc8a58b78 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:00:35.260556 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6af476-e71d-4a82-af95-75fdc8a58b78 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:00:35.261038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6af476-e71d-4a82-af95-75fdc8a58b78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:00:35.262967 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ea6af476-e71d-4a82-af95-75fdc8a58b78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:00:35.299964 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6af476-e71d-4a82-af95-75fdc8a58b78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:00:35.301396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6af476-e71d-4a82-af95-75fdc8a58b78 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:00:35.334774 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1393/2770] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:00:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:00:40.030885 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058caf38-84d4-484a-9385-60d3c94fbb73 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:00:40.031429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058caf38-84d4-484a-9385-60d3c94fbb73 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:00:40.031533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058caf38-84d4-484a-9385-60d3c94fbb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:00:40.049478 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-058caf38-84d4-484a-9385-60d3c94fbb73 None None] MFA Rules not processed for user `f2d34a97831f4a7c885ebe64aecb3669`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:00:40.050697 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-058caf38-84d4-484a-9385-60d3c94fbb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:00:40.057911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1378/2771] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:00:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:00:45.320082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=15 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:00:49.419634 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6af476-e71d-4a82-af95-75fdc8a58b78 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:00:49.429313 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6af476-e71d-4a82-af95-75fdc8a58b78 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:00:49.433254 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc16d1-ae1a-467c-99ad-0dfe270bb0b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:00:49.433923 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc16d1-ae1a-467c-99ad-0dfe270bb0b0 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:00:49.434213 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc16d1-ae1a-467c-99ad-0dfe270bb0b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:00:49.434491 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc16d1-ae1a-467c-99ad-0dfe270bb0b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:00:49.436438 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-9abc16d1-ae1a-467c-99ad-0dfe270bb0b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:00:49.451134 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc16d1-ae1a-467c-99ad-0dfe270bb0b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:00:49.452895 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc16d1-ae1a-467c-99ad-0dfe270bb0b0 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:00:49.465898 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1394/2772] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:00:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:00:50.044224 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=6 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:00:55.933684 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058caf38-84d4-484a-9385-60d3c94fbb73 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:00:55.943888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058caf38-84d4-484a-9385-60d3c94fbb73 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:00:55.947534 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92fb2d-2f5d-4809-b65b-a4c0dbd808c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:00:55.948176 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92fb2d-2f5d-4809-b65b-a4c0dbd808c8 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:00:55.948462 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92fb2d-2f5d-4809-b65b-a4c0dbd808c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:00:55.948739 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92fb2d-2f5d-4809-b65b-a4c0dbd808c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:00:55.950578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-5b92fb2d-2f5d-4809-b65b-a4c0dbd808c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:00:56.012708 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92fb2d-2f5d-4809-b65b-a4c0dbd808c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:00:56.014136 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92fb2d-2f5d-4809-b65b-a4c0dbd808c8 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:00:56.024455 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1379/2773] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:00:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 22 12:00:59.460389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=6 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:01:06.019421 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=30 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:01:24.908997 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc16d1-ae1a-467c-99ad-0dfe270bb0b0 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:01:24.917571 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc16d1-ae1a-467c-99ad-0dfe270bb0b0 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:01:24.921267 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100e2c3-17e0-4de2-94f8-db8ede1a39d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:01:24.921892 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100e2c3-17e0-4de2-94f8-db8ede1a39d4 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:01:24.922175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100e2c3-17e0-4de2-94f8-db8ede1a39d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:01:24.922453 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100e2c3-17e0-4de2-94f8-db8ede1a39d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:01:24.924226 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e100e2c3-17e0-4de2-94f8-db8ede1a39d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:01:24.933854 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100e2c3-17e0-4de2-94f8-db8ede1a39d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:01:24.935277 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100e2c3-17e0-4de2-94f8-db8ede1a39d4 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:01:24.943272 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1395/2774] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:01:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:01:34.939626 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=6 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:02:37.137971 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55b4d2-a34f-40f1-a686-547d08362dbd None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:02:37.138504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55b4d2-a34f-40f1-a686-547d08362dbd None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:02:37.138504 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55b4d2-a34f-40f1-a686-547d08362dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:02:37.157429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-6b55b4d2-a34f-40f1-a686-547d08362dbd None None] MFA Rules not processed for user `f2d34a97831f4a7c885ebe64aecb3669`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:02:37.158622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-6b55b4d2-a34f-40f1-a686-547d08362dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:02:37.165400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1380/2775] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:02:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:02:37.763163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100e2c3-17e0-4de2-94f8-db8ede1a39d4 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:02:37.764611 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e100e2c3-17e0-4de2-94f8-db8ede1a39d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:02:37.839291 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100e2c3-17e0-4de2-94f8-db8ede1a39d4 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:02:37.845304 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4637185-5dcb-4a63-82b2-eea026d977c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93630ed5e6dc4a5f958bc7cf1a68a26c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:02:37.845947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4637185-5dcb-4a63-82b2-eea026d977c9 service placement] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:02:37.846231 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4637185-5dcb-4a63-82b2-eea026d977c9 service placement] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:02:37.846508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4637185-5dcb-4a63-82b2-eea026d977c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:02:37.848302 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f4637185-5dcb-4a63-82b2-eea026d977c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:02:37.860093 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4637185-5dcb-4a63-82b2-eea026d977c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:02:37.861665 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4637185-5dcb-4a63-82b2-eea026d977c9 service placement] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:02:37.871396 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1396/2776] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:02:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:02:46.318483 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b55b4d2-a34f-40f1-a686-547d08362dbd None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:02:46.319936 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-6b55b4d2-a34f-40f1-a686-547d08362dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:02:46.337279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b55b4d2-a34f-40f1-a686-547d08362dbd None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:02:46.342783 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9cb5ee-32f4-4e51-addf-a6be6c4e56ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:02:46.343527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cb5ee-32f4-4e51-addf-a6be6c4e56ce service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:02:46.344319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cb5ee-32f4-4e51-addf-a6be6c4e56ce service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:02:46.344601 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9cb5ee-32f4-4e51-addf-a6be6c4e56ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:02:46.346403 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-0a9cb5ee-32f4-4e51-addf-a6be6c4e56ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:02:46.358604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9cb5ee-32f4-4e51-addf-a6be6c4e56ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:02:46.360328 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9cb5ee-32f4-4e51-addf-a6be6c4e56ce service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:02:46.370884 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1381/2777] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:02:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:02:47.867214 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=32 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:02:56.366468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=9 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:03:11.176593 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4637185-5dcb-4a63-82b2-eea026d977c9 service placement] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:03:11.185841 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4637185-5dcb-4a63-82b2-eea026d977c9 service placement] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:03:11.189873 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18525c6-a3f9-446b-8e00-e50682c8001d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:03:11.190501 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18525c6-a3f9-446b-8e00-e50682c8001d service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:03:11.190780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18525c6-a3f9-446b-8e00-e50682c8001d service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:03:11.191055 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18525c6-a3f9-446b-8e00-e50682c8001d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:03:11.192922 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-a18525c6-a3f9-446b-8e00-e50682c8001d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:03:11.204272 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18525c6-a3f9-446b-8e00-e50682c8001d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:03:11.206218 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18525c6-a3f9-446b-8e00-e50682c8001d service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:03:11.215756 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1397/2778] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:03:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:03:14.246101 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9cb5ee-32f4-4e51-addf-a6be6c4e56ce service nova] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:03:14.257635 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9cb5ee-32f4-4e51-addf-a6be6c4e56ce service nova] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:03:14.262658 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f224323-b3d3-4838-a11c-5b3a91444262 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93630ed5e6dc4a5f958bc7cf1a68a26c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:03:14.263303 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f224323-b3d3-4838-a11c-5b3a91444262 service placement] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:03:14.263618 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f224323-b3d3-4838-a11c-5b3a91444262 service placement] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:03:14.263893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f224323-b3d3-4838-a11c-5b3a91444262 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:03:14.265663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-0f224323-b3d3-4838-a11c-5b3a91444262 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:03:14.277982 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f224323-b3d3-4838-a11c-5b3a91444262 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:03:14.279751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f224323-b3d3-4838-a11c-5b3a91444262 service placement] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:03:14.292721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1382/2779] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:03:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:03:17.055914 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18525c6-a3f9-446b-8e00-e50682c8001d service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:03:17.067658 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18525c6-a3f9-446b-8e00-e50682c8001d service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:03:17.073261 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe59efc-144c-4a2b-9b20-a5066c40dd1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93630ed5e6dc4a5f958bc7cf1a68a26c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:03:17.073906 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe59efc-144c-4a2b-9b20-a5066c40dd1c service placement] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:03:17.074188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe59efc-144c-4a2b-9b20-a5066c40dd1c service placement] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:03:17.074463 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe59efc-144c-4a2b-9b20-a5066c40dd1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:03:17.076338 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-abe59efc-144c-4a2b-9b20-a5066c40dd1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:03:17.089883 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe59efc-144c-4a2b-9b20-a5066c40dd1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:03:17.091748 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe59efc-144c-4a2b-9b20-a5066c40dd1c service placement] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:03:17.103714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1398/2780] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:03:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:03:17.122964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f224323-b3d3-4838-a11c-5b3a91444262 service placement] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:03:17.124467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-0f224323-b3d3-4838-a11c-5b3a91444262 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:03:17.141049 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f224323-b3d3-4838-a11c-5b3a91444262 service placement] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:03:17.146744 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539541c-c3a8-4b79-af9a-4ac54d2bb889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:03:17.148037 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539541c-c3a8-4b79-af9a-4ac54d2bb889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:03:17.148385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539541c-c3a8-4b79-af9a-4ac54d2bb889 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:03:17.148665 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539541c-c3a8-4b79-af9a-4ac54d2bb889 admin admin] PATH_INFO: `/v3/users/239de565f79f474bb21699fd9adc96c4` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:03:17.150111 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b539541c-c3a8-4b79-af9a-4ac54d2bb889 admin admin] RBAC: Authorizing `identity:delete_user(user_id=239de565f79f474bb21699fd9adc96c4)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:03:17.152352 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b539541c-c3a8-4b79-af9a-4ac54d2bb889 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:03:17.205443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b539541c-c3a8-4b79-af9a-4ac54d2bb889 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '239de565f79f474bb21699fd9adc96c4'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:17.217561 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b539541c-c3a8-4b79-af9a-4ac54d2bb889 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '239de565f79f474bb21699fd9adc96c4'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:17.224893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b539541c-c3a8-4b79-af9a-4ac54d2bb889 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '239de565f79f474bb21699fd9adc96c4'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:17.225801 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b539541c-c3a8-4b79-af9a-4ac54d2bb889 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '239de565f79f474bb21699fd9adc96c4'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:17.233695 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b539541c-c3a8-4b79-af9a-4ac54d2bb889 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '239de565f79f474bb21699fd9adc96c4'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:17.240239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1383/2781] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 12:03:17 2022] DELETE /identity/v3/users/239de565f79f474bb21699fd9adc96c4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 12:03:17.408968 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe59efc-144c-4a2b-9b20-a5066c40dd1c service placement] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:03:17.421320 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe59efc-144c-4a2b-9b20-a5066c40dd1c service placement] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:03:17.480000 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b77a61-434e-485f-aa01-896decc82712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:03:17.480794 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b77a61-434e-485f-aa01-896decc82712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:03:17.481079 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b77a61-434e-485f-aa01-896decc82712 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:03:17.481358 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b77a61-434e-485f-aa01-896decc82712 admin admin] PATH_INFO: `/v3/projects/edbd324693e24bcdbf41c207d29d7e2e` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:03:17.482542 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b77a61-434e-485f-aa01-896decc82712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edbd324693e24bcdbf41c207d29d7e2e)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:03:17.484490 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b77a61-434e-485f-aa01-896decc82712 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:03:17.530807 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-84b77a61-434e-485f-aa01-896decc82712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edbd324693e24bcdbf41c207d29d7e2e', 'request_id': 'req-84b77a61-434e-485f-aa01-896decc82712'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:17.537106 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-84b77a61-434e-485f-aa01-896decc82712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edbd324693e24bcdbf41c207d29d7e2e', 'request_id': 'req-84b77a61-434e-485f-aa01-896decc82712'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:17.544837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-84b77a61-434e-485f-aa01-896decc82712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edbd324693e24bcdbf41c207d29d7e2e', 'request_id': 'req-84b77a61-434e-485f-aa01-896decc82712'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:17.556182 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-84b77a61-434e-485f-aa01-896decc82712 admin admin] The token cache is being invalidate because project edbd324693e24bcdbf41c207d29d7e2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 12:03:17.556467 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-84b77a61-434e-485f-aa01-896decc82712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:17.557096 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-84b77a61-434e-485f-aa01-896decc82712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:17.558645 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1399/2782] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 12:03:17 2022] DELETE /identity/v3/projects/edbd324693e24bcdbf41c207d29d7e2e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:03:17.564011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539541c-c3a8-4b79-af9a-4ac54d2bb889 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:03:17.566094 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b539541c-c3a8-4b79-af9a-4ac54d2bb889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:03:17.579400 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b539541c-c3a8-4b79-af9a-4ac54d2bb889 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:03:17.636663 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eedda5-aa64-47ba-823b-9f2a2d3cec84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:03:17.637569 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eedda5-aa64-47ba-823b-9f2a2d3cec84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:03:17.637854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eedda5-aa64-47ba-823b-9f2a2d3cec84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:03:17.638135 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eedda5-aa64-47ba-823b-9f2a2d3cec84 admin admin] PATH_INFO: `/v3/users/258c84e10fd3449794752d64f56d41ec` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:03:17.639397 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eedda5-aa64-47ba-823b-9f2a2d3cec84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=258c84e10fd3449794752d64f56d41ec)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:03:17.641321 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eedda5-aa64-47ba-823b-9f2a2d3cec84 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:03:17.688235 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-25eedda5-aa64-47ba-823b-9f2a2d3cec84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '258c84e10fd3449794752d64f56d41ec'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:17.698207 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:03:17.700808 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-25eedda5-aa64-47ba-823b-9f2a2d3cec84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '258c84e10fd3449794752d64f56d41ec'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:17.708613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-25eedda5-aa64-47ba-823b-9f2a2d3cec84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '258c84e10fd3449794752d64f56d41ec'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:17.709642 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-25eedda5-aa64-47ba-823b-9f2a2d3cec84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '258c84e10fd3449794752d64f56d41ec'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:17.716524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-25eedda5-aa64-47ba-823b-9f2a2d3cec84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '258c84e10fd3449794752d64f56d41ec'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:17.722893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1384/2783] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 12:03:17 2022] DELETE /identity/v3/users/258c84e10fd3449794752d64f56d41ec => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:03:17.897841 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b77a61-434e-485f-aa01-896decc82712 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:03:17.899369 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-84b77a61-434e-485f-aa01-896decc82712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:03:17.913103 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b77a61-434e-485f-aa01-896decc82712 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:03:17.970752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75fe79-2b11-4ce2-b3fc-66f1a936e8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:03:17.971661 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75fe79-2b11-4ce2-b3fc-66f1a936e8f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:03:17.971951 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75fe79-2b11-4ce2-b3fc-66f1a936e8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:03:17.972232 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75fe79-2b11-4ce2-b3fc-66f1a936e8f0 admin admin] PATH_INFO: `/v3/projects/a71966a6b5ca4471892906c483adf73d` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:03:17.973508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75fe79-2b11-4ce2-b3fc-66f1a936e8f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a71966a6b5ca4471892906c483adf73d)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:03:17.975470 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75fe79-2b11-4ce2-b3fc-66f1a936e8f0 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:03:17.981657 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:03:18.022790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-fb75fe79-2b11-4ce2-b3fc-66f1a936e8f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a71966a6b5ca4471892906c483adf73d', 'request_id': 'req-fb75fe79-2b11-4ce2-b3fc-66f1a936e8f0'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:18.028789 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-fb75fe79-2b11-4ce2-b3fc-66f1a936e8f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a71966a6b5ca4471892906c483adf73d', 'request_id': 'req-fb75fe79-2b11-4ce2-b3fc-66f1a936e8f0'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:18.036052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-fb75fe79-2b11-4ce2-b3fc-66f1a936e8f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a71966a6b5ca4471892906c483adf73d', 'request_id': 'req-fb75fe79-2b11-4ce2-b3fc-66f1a936e8f0'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:18.046102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-fb75fe79-2b11-4ce2-b3fc-66f1a936e8f0 admin admin] The token cache is being invalidate because project a71966a6b5ca4471892906c483adf73d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 12:03:18.046389 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-fb75fe79-2b11-4ce2-b3fc-66f1a936e8f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:18.047002 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-fb75fe79-2b11-4ce2-b3fc-66f1a936e8f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:03:18.048540 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1400/2784] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 12:03:17 2022] DELETE /identity/v3/projects/a71966a6b5ca4471892906c483adf73d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:03:23.500213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eedda5-aa64-47ba-823b-9f2a2d3cec84 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:03:23.501629 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-25eedda5-aa64-47ba-823b-9f2a2d3cec84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:03:23.572414 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eedda5-aa64-47ba-823b-9f2a2d3cec84 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:03:23.629393 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b309f8-4f3c-4f63-8eab-62824ac902d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93630ed5e6dc4a5f958bc7cf1a68a26c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:03:23.630036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b309f8-4f3c-4f63-8eab-62824ac902d5 service placement] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:03:23.630320 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b309f8-4f3c-4f63-8eab-62824ac902d5 service placement] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:03:23.630602 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b309f8-4f3c-4f63-8eab-62824ac902d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:03:23.632402 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b9b309f8-4f3c-4f63-8eab-62824ac902d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:03:23.643339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b309f8-4f3c-4f63-8eab-62824ac902d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:03:23.644921 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b309f8-4f3c-4f63-8eab-62824ac902d5 service placement] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:03:23.654840 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1385/2785] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:03:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:03:28.043137 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:03:33.650707 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:DELETE=2,keystone:SELECT=65,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:04:04.535919 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75fe79-2b11-4ce2-b3fc-66f1a936e8f0 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:04:04.537459 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-fb75fe79-2b11-4ce2-b3fc-66f1a936e8f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:04:04.610059 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75fe79-2b11-4ce2-b3fc-66f1a936e8f0 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:04:04.616164 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddf0d3-7078-4312-b551-a68cf9a90322 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:04:04.616804 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddf0d3-7078-4312-b551-a68cf9a90322 service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:04:04.617087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddf0d3-7078-4312-b551-a68cf9a90322 service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:04:04.617364 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddf0d3-7078-4312-b551-a68cf9a90322 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:04:04.619112 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-a8ddf0d3-7078-4312-b551-a68cf9a90322 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:04:04.675523 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ddf0d3-7078-4312-b551-a68cf9a90322 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:04:04.677115 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ddf0d3-7078-4312-b551-a68cf9a90322 service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:04:04.685812 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1401/2786] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:04:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:04:14.681792 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=56 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:04:18.192604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b309f8-4f3c-4f63-8eab-62824ac902d5 service placement] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:04:18.205407 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b309f8-4f3c-4f63-8eab-62824ac902d5 service placement] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:04:18.214007 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436807d-2127-42a8-baa7-8eda176d4c3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:04:18.214669 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436807d-2127-42a8-baa7-8eda176d4c3b service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:04:18.215429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436807d-2127-42a8-baa7-8eda176d4c3b service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:04:18.215429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436807d-2127-42a8-baa7-8eda176d4c3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:04:18.217092 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2436807d-2127-42a8-baa7-8eda176d4c3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:04:18.230412 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436807d-2127-42a8-baa7-8eda176d4c3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:04:18.232372 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2436807d-2127-42a8-baa7-8eda176d4c3b service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:04:18.245167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1386/2787] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:04:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:04:21.124940 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e987d-7a45-421a-a900-2f95785f6371 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:04:21.125441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e987d-7a45-421a-a900-2f95785f6371 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:04:21.125549 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e987d-7a45-421a-a900-2f95785f6371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:04:21.142622 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-af1e987d-7a45-421a-a900-2f95785f6371 None None] MFA Rules not processed for user `f2d34a97831f4a7c885ebe64aecb3669`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:04:21.143833 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-af1e987d-7a45-421a-a900-2f95785f6371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:04:21.151420 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1402/2788] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:04:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:04:21.454793 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436807d-2127-42a8-baa7-8eda176d4c3b service nova] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:04:21.456262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2436807d-2127-42a8-baa7-8eda176d4c3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:04:21.511578 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436807d-2127-42a8-baa7-8eda176d4c3b service nova] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:04:21.515133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76bb45-7df9-4cfa-a233-0df0b6bd43f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:04:21.515810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76bb45-7df9-4cfa-a233-0df0b6bd43f0 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:04:21.516093 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76bb45-7df9-4cfa-a233-0df0b6bd43f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:04:21.516370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76bb45-7df9-4cfa-a233-0df0b6bd43f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:04:21.518108 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-0b76bb45-7df9-4cfa-a233-0df0b6bd43f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:04:21.527907 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76bb45-7df9-4cfa-a233-0df0b6bd43f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:04:21.529318 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b76bb45-7df9-4cfa-a233-0df0b6bd43f0 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:04:21.537334 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1387/2789] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:04:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:04:31.137897 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:04:31.533780 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=32 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:04:32.187876 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e987d-7a45-421a-a900-2f95785f6371 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:04:32.189468 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-af1e987d-7a45-421a-a900-2f95785f6371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:04:32.205901 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e987d-7a45-421a-a900-2f95785f6371 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:04:32.211459 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25e57f-11a2-41f1-b63e-f0965107e0d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:04:32.212091 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25e57f-11a2-41f1-b63e-f0965107e0d0 service nova] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:04:32.212376 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25e57f-11a2-41f1-b63e-f0965107e0d0 service nova] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:04:32.212659 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25e57f-11a2-41f1-b63e-f0965107e0d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:04:32.214448 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-dd25e57f-11a2-41f1-b63e-f0965107e0d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:04:32.227372 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25e57f-11a2-41f1-b63e-f0965107e0d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:04:32.228969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25e57f-11a2-41f1-b63e-f0965107e0d0 service nova] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:04:32.240324 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1403/2790] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:04:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:04:42.235006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=6 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:05:48.678702 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76bb45-7df9-4cfa-a233-0df0b6bd43f0 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:05:48.680139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-0b76bb45-7df9-4cfa-a233-0df0b6bd43f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:05:48.755704 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76bb45-7df9-4cfa-a233-0df0b6bd43f0 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:05:48.759825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eca401-9e86-4891-9ec6-e75717913201 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:05:48.760464 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eca401-9e86-4891-9ec6-e75717913201 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:05:48.760753 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eca401-9e86-4891-9ec6-e75717913201 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:05:48.761028 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eca401-9e86-4891-9ec6-e75717913201 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:05:48.762787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b0eca401-9e86-4891-9ec6-e75717913201 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:05:48.776680 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eca401-9e86-4891-9ec6-e75717913201 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:05:48.778084 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eca401-9e86-4891-9ec6-e75717913201 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:05:48.788877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1388/2791] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:05:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:05:56.501837 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25e57f-11a2-41f1-b63e-f0965107e0d0 service nova] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:05:56.503223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-dd25e57f-11a2-41f1-b63e-f0965107e0d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:05:56.514570 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25e57f-11a2-41f1-b63e-f0965107e0d0 service nova] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:05:56.518059 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82194dbb-e58e-4de2-9cba-7471fb549b04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:05:56.518683 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82194dbb-e58e-4de2-9cba-7471fb549b04 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:05:56.518965 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82194dbb-e58e-4de2-9cba-7471fb549b04 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:05:56.519241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82194dbb-e58e-4de2-9cba-7471fb549b04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:05:56.520969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-82194dbb-e58e-4de2-9cba-7471fb549b04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:05:56.534448 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82194dbb-e58e-4de2-9cba-7471fb549b04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:05:56.535890 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82194dbb-e58e-4de2-9cba-7471fb549b04 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:05:56.546993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1404/2792] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:05:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:05:58.784080 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=35 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:06:06.541719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=6 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:06:06.838843 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eca401-9e86-4891-9ec6-e75717913201 service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:06.848095 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eca401-9e86-4891-9ec6-e75717913201 service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:06.851660 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28f763-8d05-41ed-9ae1-20137dd23637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:06.852311 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28f763-8d05-41ed-9ae1-20137dd23637 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:06.852593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28f763-8d05-41ed-9ae1-20137dd23637 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:06.852870 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28f763-8d05-41ed-9ae1-20137dd23637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:06.854640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-8f28f763-8d05-41ed-9ae1-20137dd23637 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:06.917020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28f763-8d05-41ed-9ae1-20137dd23637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:06.918441 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28f763-8d05-41ed-9ae1-20137dd23637 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:06.927932 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1389/2793] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:06:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:06:10.027549 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82194dbb-e58e-4de2-9cba-7471fb549b04 service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:10.031026 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-82194dbb-e58e-4de2-9cba-7471fb549b04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:10.053574 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82194dbb-e58e-4de2-9cba-7471fb549b04 service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:10.060276 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c950ba-f133-406b-9abd-bd6b11404cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:10.061190 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c950ba-f133-406b-9abd-bd6b11404cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:10.061559 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c950ba-f133-406b-9abd-bd6b11404cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:10.061798 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c950ba-f133-406b-9abd-bd6b11404cdd admin admin] PATH_INFO: `/v3/users/f2d34a97831f4a7c885ebe64aecb3669` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:10.063468 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c950ba-f133-406b-9abd-bd6b11404cdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2d34a97831f4a7c885ebe64aecb3669)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:10.065936 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c950ba-f133-406b-9abd-bd6b11404cdd admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:10.122220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f9c950ba-f133-406b-9abd-bd6b11404cdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2d34a97831f4a7c885ebe64aecb3669'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:10.135135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f9c950ba-f133-406b-9abd-bd6b11404cdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2d34a97831f4a7c885ebe64aecb3669'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:10.142821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f9c950ba-f133-406b-9abd-bd6b11404cdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2d34a97831f4a7c885ebe64aecb3669'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:10.150096 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f9c950ba-f133-406b-9abd-bd6b11404cdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2d34a97831f4a7c885ebe64aecb3669'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:10.150764 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-f9c950ba-f133-406b-9abd-bd6b11404cdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2d34a97831f4a7c885ebe64aecb3669'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:10.156668 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1405/2794] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 12:06:10 2022] DELETE /identity/v3/users/f2d34a97831f4a7c885ebe64aecb3669 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:06:10.332540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28f763-8d05-41ed-9ae1-20137dd23637 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:10.343183 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28f763-8d05-41ed-9ae1-20137dd23637 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:10.405841 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca8b2a-8e8e-4db7-acf8-66a349c5ca13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:10.406691 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca8b2a-8e8e-4db7-acf8-66a349c5ca13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:10.406977 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca8b2a-8e8e-4db7-acf8-66a349c5ca13 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:10.407253 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca8b2a-8e8e-4db7-acf8-66a349c5ca13 admin admin] PATH_INFO: `/v3/projects/897ec8bf954c41648da9cac00bc44749` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:10.408465 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca8b2a-8e8e-4db7-acf8-66a349c5ca13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=897ec8bf954c41648da9cac00bc44749)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:10.410361 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca8b2a-8e8e-4db7-acf8-66a349c5ca13 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:10.455897 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-84ca8b2a-8e8e-4db7-acf8-66a349c5ca13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '897ec8bf954c41648da9cac00bc44749', 'request_id': 'req-84ca8b2a-8e8e-4db7-acf8-66a349c5ca13'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:10.463195 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-84ca8b2a-8e8e-4db7-acf8-66a349c5ca13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '897ec8bf954c41648da9cac00bc44749', 'request_id': 'req-84ca8b2a-8e8e-4db7-acf8-66a349c5ca13'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:10.469286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-84ca8b2a-8e8e-4db7-acf8-66a349c5ca13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '897ec8bf954c41648da9cac00bc44749', 'request_id': 'req-84ca8b2a-8e8e-4db7-acf8-66a349c5ca13'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:10.479768 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-84ca8b2a-8e8e-4db7-acf8-66a349c5ca13 admin admin] The token cache is being invalidate because project 897ec8bf954c41648da9cac00bc44749 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 12:06:10.480088 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-84ca8b2a-8e8e-4db7-acf8-66a349c5ca13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:10.480733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-84ca8b2a-8e8e-4db7-acf8-66a349c5ca13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:10.482248 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1390/2795] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 12:06:10 2022] DELETE /identity/v3/projects/897ec8bf954c41648da9cac00bc44749 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:06:10.487298 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1bca8-149a-45dc-a2fe-6cf877be322b None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:10.487743 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1bca8-149a-45dc-a2fe-6cf877be322b None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:10.488019 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1bca8-149a-45dc-a2fe-6cf877be322b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:10.505533 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-94e1bca8-149a-45dc-a2fe-6cf877be322b None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:06:10.506724 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-94e1bca8-149a-45dc-a2fe-6cf877be322b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:10.564083 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1406/2796] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:06:10.571662 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b0490-c2f3-428a-9446-d9b3d06bab9f None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:10.571956 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b0490-c2f3-428a-9446-d9b3d06bab9f None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:10.572297 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b0490-c2f3-428a-9446-d9b3d06bab9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:10.591054 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-fb6b0490-c2f3-428a-9446-d9b3d06bab9f None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:06:10.592301 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-fb6b0490-c2f3-428a-9446-d9b3d06bab9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:10.600171 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1391/2797] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:06:10.604675 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1bca8-149a-45dc-a2fe-6cf877be322b None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:10.614135 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e1bca8-149a-45dc-a2fe-6cf877be322b None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:10.618677 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8165c7d-f257-4706-86a7-71d73d8ce372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:10.619992 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8165c7d-f257-4706-86a7-71d73d8ce372 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:10.620287 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8165c7d-f257-4706-86a7-71d73d8ce372 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:10.620565 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8165c7d-f257-4706-86a7-71d73d8ce372 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:10.621289 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8165c7d-f257-4706-86a7-71d73d8ce372 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 12:06:10.621595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8165c7d-f257-4706-86a7-71d73d8ce372 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:10.623189 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8165c7d-f257-4706-86a7-71d73d8ce372 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:10.634435 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1407/2798] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 12:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 12:06:10.638521 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b0490-c2f3-428a-9446-d9b3d06bab9f None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:10.647966 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b0490-c2f3-428a-9446-d9b3d06bab9f None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:10.653394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a85ea1-d368-4e42-99d4-1bf119db8d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:10.654036 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a85ea1-d368-4e42-99d4-1bf119db8d42 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:10.654323 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a85ea1-d368-4e42-99d4-1bf119db8d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:10.654600 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a85ea1-d368-4e42-99d4-1bf119db8d42 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:10.655290 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a85ea1-d368-4e42-99d4-1bf119db8d42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:10.657206 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a85ea1-d368-4e42-99d4-1bf119db8d42 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:10.671593 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:06:10.692696 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1392/2799] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 12:06:10 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 12:06:10.696776 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8165c7d-f257-4706-86a7-71d73d8ce372 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:10.706665 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8165c7d-f257-4706-86a7-71d73d8ce372 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:10.761117 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f9f07-8f95-4b6b-a9dd-04cd2b11a22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:10.761720 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f9f07-8f95-4b6b-a9dd-04cd2b11a22d admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:10.762032 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f9f07-8f95-4b6b-a9dd-04cd2b11a22d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:10.762313 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0f9f07-8f95-4b6b-a9dd-04cd2b11a22d admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:10.763011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f9f07-8f95-4b6b-a9dd-04cd2b11a22d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:10.764977 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0f9f07-8f95-4b6b-a9dd-04cd2b11a22d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:10.771519 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:06:10.786332 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1408/2800] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 12:06:10 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 12:06:10.790361 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a85ea1-d368-4e42-99d4-1bf119db8d42 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:10.801034 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a85ea1-d368-4e42-99d4-1bf119db8d42 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:10.805618 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757c59a-e0b0-4e7a-b4cb-7a83cddb1522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:10.806265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757c59a-e0b0-4e7a-b4cb-7a83cddb1522 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:10.806547 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757c59a-e0b0-4e7a-b4cb-7a83cddb1522 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:10.806889 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757c59a-e0b0-4e7a-b4cb-7a83cddb1522 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:10.807647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757c59a-e0b0-4e7a-b4cb-7a83cddb1522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:10.809225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757c59a-e0b0-4e7a-b4cb-7a83cddb1522 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:10.818900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1393/2801] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:06:10.822676 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f9f07-8f95-4b6b-a9dd-04cd2b11a22d admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:10.832480 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0f9f07-8f95-4b6b-a9dd-04cd2b11a22d admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:10.837077 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5fc41-28dc-45f9-9c58-cab2e5ceb446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:10.837725 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5fc41-28dc-45f9-9c58-cab2e5ceb446 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:10.838537 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5fc41-28dc-45f9-9c58-cab2e5ceb446 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:10.838821 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5fc41-28dc-45f9-9c58-cab2e5ceb446 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:10.839576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5fc41-28dc-45f9-9c58-cab2e5ceb446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:10.841153 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5fc41-28dc-45f9-9c58-cab2e5ceb446 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:10.852042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1409/2802] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:06:10.856484 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757c59a-e0b0-4e7a-b4cb-7a83cddb1522 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:10.870026 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757c59a-e0b0-4e7a-b4cb-7a83cddb1522 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:10.874658 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120391b-f10f-4e67-8751-b97d97e3e32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:10.875553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120391b-f10f-4e67-8751-b97d97e3e32c admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:10.875836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120391b-f10f-4e67-8751-b97d97e3e32c admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:10.876110 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d120391b-f10f-4e67-8751-b97d97e3e32c admin admin] PATH_INFO: `/v3/projects/509e8386f2294976a3dee85c162c6111/users/f81832aafe4f4ef89607c30c0423e553/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:10.890507 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120391b-f10f-4e67-8751-b97d97e3e32c admin admin] RBAC: Authorizing `identity:create_grant(project_id=509e8386f2294976a3dee85c162c6111, user_id=f81832aafe4f4ef89607c30c0423e553, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:10.893259 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d120391b-f10f-4e67-8751-b97d97e3e32c admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:10.906622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1394/2803] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 12:06:10 2022] PUT /identity/v3/projects/509e8386f2294976a3dee85c162c6111/users/f81832aafe4f4ef89607c30c0423e553/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 12:06:10.917206 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78508c-c56b-4792-91c3-c7ef56cebafd None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:10.917501 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78508c-c56b-4792-91c3-c7ef56cebafd None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:10.917779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78508c-c56b-4792-91c3-c7ef56cebafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:10.944069 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-2b78508c-c56b-4792-91c3-c7ef56cebafd None None] MFA Rules not processed for user `f81832aafe4f4ef89607c30c0423e553`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:06:10.945280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-2b78508c-c56b-4792-91c3-c7ef56cebafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:11.008212 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1410/2804] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:06:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:06:11.020003 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120391b-f10f-4e67-8751-b97d97e3e32c admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:11.021648 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-d120391b-f10f-4e67-8751-b97d97e3e32c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:11.092430 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d120391b-f10f-4e67-8751-b97d97e3e32c admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:11.097901 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d38086-8f47-4e50-bb5b-088229e49e0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:11.098538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d38086-8f47-4e50-bb5b-088229e49e0c service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:11.098825 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d38086-8f47-4e50-bb5b-088229e49e0c service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:11.099103 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d38086-8f47-4e50-bb5b-088229e49e0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:11.107249 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d38086-8f47-4e50-bb5b-088229e49e0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:11.108836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d38086-8f47-4e50-bb5b-088229e49e0c service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:11.117456 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1395/2805] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:06:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:06:19.395636 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78508c-c56b-4792-91c3-c7ef56cebafd None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:19.455952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78508c-c56b-4792-91c3-c7ef56cebafd None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:19.460509 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03275f6a-086e-4a3a-bfc8-fa8fd4c9e0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:19.461368 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03275f6a-086e-4a3a-bfc8-fa8fd4c9e0f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:19.461654 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03275f6a-086e-4a3a-bfc8-fa8fd4c9e0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:19.461932 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03275f6a-086e-4a3a-bfc8-fa8fd4c9e0f4 admin admin] PATH_INFO: `/v3/users/f81832aafe4f4ef89607c30c0423e553` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:19.463705 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03275f6a-086e-4a3a-bfc8-fa8fd4c9e0f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f81832aafe4f4ef89607c30c0423e553)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:19.465616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03275f6a-086e-4a3a-bfc8-fa8fd4c9e0f4 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:19.511765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-03275f6a-086e-4a3a-bfc8-fa8fd4c9e0f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f81832aafe4f4ef89607c30c0423e553'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:19.523545 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-03275f6a-086e-4a3a-bfc8-fa8fd4c9e0f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f81832aafe4f4ef89607c30c0423e553'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:19.527993 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:06:19.532751 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-03275f6a-086e-4a3a-bfc8-fa8fd4c9e0f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f81832aafe4f4ef89607c30c0423e553'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:19.541526 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-03275f6a-086e-4a3a-bfc8-fa8fd4c9e0f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f81832aafe4f4ef89607c30c0423e553'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:19.542153 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-03275f6a-086e-4a3a-bfc8-fa8fd4c9e0f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f81832aafe4f4ef89607c30c0423e553'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:19.547983 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1411/2806] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 12:06:19 2022] DELETE /identity/v3/users/f81832aafe4f4ef89607c30c0423e553 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:06:19.552581 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a272b-b493-41bf-9124-dd390fb8ac25 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:19.553041 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a272b-b493-41bf-9124-dd390fb8ac25 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:19.553138 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a272b-b493-41bf-9124-dd390fb8ac25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:19.570240 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-151a272b-b493-41bf-9124-dd390fb8ac25 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:06:19.571523 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-151a272b-b493-41bf-9124-dd390fb8ac25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:19.630968 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1396/2807] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:06:19.640990 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03275f6a-086e-4a3a-bfc8-fa8fd4c9e0f4 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:19.642388 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-03275f6a-086e-4a3a-bfc8-fa8fd4c9e0f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:19.699617 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03275f6a-086e-4a3a-bfc8-fa8fd4c9e0f4 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:19.703448 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a9b7e-07eb-430c-9d81-6e82acd8df05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:19.704091 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a9b7e-07eb-430c-9d81-6e82acd8df05 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:19.704369 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a9b7e-07eb-430c-9d81-6e82acd8df05 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:19.704644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a9b7e-07eb-430c-9d81-6e82acd8df05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:19.714108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a9b7e-07eb-430c-9d81-6e82acd8df05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:19.715558 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a9b7e-07eb-430c-9d81-6e82acd8df05 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:19.724766 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1412/2808] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:06:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:06:19.987004 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a272b-b493-41bf-9124-dd390fb8ac25 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:19.988563 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-151a272b-b493-41bf-9124-dd390fb8ac25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:20.002699 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a272b-b493-41bf-9124-dd390fb8ac25 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:20.007443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf6345-3912-4ce9-bd72-68ec809f781a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:20.008335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf6345-3912-4ce9-bd72-68ec809f781a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:20.008623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf6345-3912-4ce9-bd72-68ec809f781a admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:20.008903 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf6345-3912-4ce9-bd72-68ec809f781a admin admin] PATH_INFO: `/v3/projects/509e8386f2294976a3dee85c162c6111` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:20.010901 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf6345-3912-4ce9-bd72-68ec809f781a admin admin] RBAC: Authorizing `identity:delete_project(project_id=509e8386f2294976a3dee85c162c6111)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:20.012759 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf6345-3912-4ce9-bd72-68ec809f781a admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:20.049633 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:06:20.064348 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b3bf6345-3912-4ce9-bd72-68ec809f781a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '509e8386f2294976a3dee85c162c6111', 'request_id': 'req-b3bf6345-3912-4ce9-bd72-68ec809f781a'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:20.071438 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b3bf6345-3912-4ce9-bd72-68ec809f781a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '509e8386f2294976a3dee85c162c6111', 'request_id': 'req-b3bf6345-3912-4ce9-bd72-68ec809f781a'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:20.077411 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b3bf6345-3912-4ce9-bd72-68ec809f781a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '509e8386f2294976a3dee85c162c6111', 'request_id': 'req-b3bf6345-3912-4ce9-bd72-68ec809f781a'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:20.087459 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b3bf6345-3912-4ce9-bd72-68ec809f781a admin admin] The token cache is being invalidate because project 509e8386f2294976a3dee85c162c6111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 12:06:20.087749 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b3bf6345-3912-4ce9-bd72-68ec809f781a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:20.088403 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b3bf6345-3912-4ce9-bd72-68ec809f781a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:20.089921 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1397/2809] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 12:06:19 2022] DELETE /identity/v3/projects/509e8386f2294976a3dee85c162c6111 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:06:20.094680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7b8df-c677-4e37-968f-886ebafcee33 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:20.094974 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7b8df-c677-4e37-968f-886ebafcee33 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:20.095251 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7b8df-c677-4e37-968f-886ebafcee33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:20.112848 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-41f7b8df-c677-4e37-968f-886ebafcee33 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:06:20.114041 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-41f7b8df-c677-4e37-968f-886ebafcee33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:20.171840 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1413/2810] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:06:20.180654 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bee18f-0a03-4b34-a9ec-22a5280ea082 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:20.180942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bee18f-0a03-4b34-a9ec-22a5280ea082 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:20.181221 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bee18f-0a03-4b34-a9ec-22a5280ea082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:20.199493 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-c5bee18f-0a03-4b34-a9ec-22a5280ea082 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:06:20.200715 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c5bee18f-0a03-4b34-a9ec-22a5280ea082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:20.208512 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1398/2811] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:06:20.212638 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7b8df-c677-4e37-968f-886ebafcee33 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:20.222306 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7b8df-c677-4e37-968f-886ebafcee33 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:20.226904 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80e046-afd6-4ec4-a6d5-5837a747e20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:20.227699 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80e046-afd6-4ec4-a6d5-5837a747e20e admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:20.227981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80e046-afd6-4ec4-a6d5-5837a747e20e admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:20.228764 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80e046-afd6-4ec4-a6d5-5837a747e20e admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:20.229482 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80e046-afd6-4ec4-a6d5-5837a747e20e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 12:06:20.229787 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80e046-afd6-4ec4-a6d5-5837a747e20e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:20.231442 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80e046-afd6-4ec4-a6d5-5837a747e20e admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:20.242969 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1414/2812] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 12:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 12:06:20.247319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bee18f-0a03-4b34-a9ec-22a5280ea082 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:20.256511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bee18f-0a03-4b34-a9ec-22a5280ea082 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:20.261091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c1ba7-f8e9-4dec-bf4c-bf737703d886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:20.261721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c1ba7-f8e9-4dec-bf4c-bf737703d886 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:20.262003 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c1ba7-f8e9-4dec-bf4c-bf737703d886 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:20.262279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c1ba7-f8e9-4dec-bf4c-bf737703d886 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:20.262962 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c1ba7-f8e9-4dec-bf4c-bf737703d886 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:20.264877 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c1ba7-f8e9-4dec-bf4c-bf737703d886 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:20.289346 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1399/2813] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 12:06:20 2022] POST /identity/v3/projects => generated 363 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 12:06:20.293815 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80e046-afd6-4ec4-a6d5-5837a747e20e admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:20.303607 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80e046-afd6-4ec4-a6d5-5837a747e20e admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:20.360386 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf920de-247f-4b2b-89cb-693164be3ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:20.360972 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf920de-247f-4b2b-89cb-693164be3ffb admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:20.361255 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf920de-247f-4b2b-89cb-693164be3ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:20.361529 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf920de-247f-4b2b-89cb-693164be3ffb admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:20.362201 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf920de-247f-4b2b-89cb-693164be3ffb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:20.364136 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf920de-247f-4b2b-89cb-693164be3ffb admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:20.382834 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1415/2814] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 12:06:20 2022] POST /identity/v3/users => generated 282 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 12:06:20.387413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c1ba7-f8e9-4dec-bf4c-bf737703d886 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:20.397406 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c1ba7-f8e9-4dec-bf4c-bf737703d886 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:20.402867 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2aa26-b91f-43a1-aa96-1e8b718c3dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:20.403551 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2aa26-b91f-43a1-aa96-1e8b718c3dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:20.403836 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2aa26-b91f-43a1-aa96-1e8b718c3dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:20.404109 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2aa26-b91f-43a1-aa96-1e8b718c3dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:20.404822 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2aa26-b91f-43a1-aa96-1e8b718c3dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:20.406384 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2aa26-b91f-43a1-aa96-1e8b718c3dc9 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:20.416566 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1400/2815] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:06:20.420310 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf920de-247f-4b2b-89cb-693164be3ffb admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:20.430319 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf920de-247f-4b2b-89cb-693164be3ffb admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:20.435497 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16291d-1362-4db5-8d71-ef463a5d71df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:20.436146 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16291d-1362-4db5-8d71-ef463a5d71df admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:20.436433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16291d-1362-4db5-8d71-ef463a5d71df admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:20.436716 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16291d-1362-4db5-8d71-ef463a5d71df admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:20.437458 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16291d-1362-4db5-8d71-ef463a5d71df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:20.439030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16291d-1362-4db5-8d71-ef463a5d71df admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:20.440271 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:06:20.451072 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1416/2816] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:06:20.455340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2aa26-b91f-43a1-aa96-1e8b718c3dc9 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:20.465126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2aa26-b91f-43a1-aa96-1e8b718c3dc9 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:20.469892 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637221cb-bdbc-4401-b72e-a738b7fc1ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:20.470757 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637221cb-bdbc-4401-b72e-a738b7fc1ebb admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:20.471038 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637221cb-bdbc-4401-b72e-a738b7fc1ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:20.471314 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637221cb-bdbc-4401-b72e-a738b7fc1ebb admin admin] PATH_INFO: `/v3/projects/125fdf6902704f40afe5f94a3f3bfb80/users/0be6ec7b80324dac830727e7297115ae/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:20.485463 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637221cb-bdbc-4401-b72e-a738b7fc1ebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=125fdf6902704f40afe5f94a3f3bfb80, user_id=0be6ec7b80324dac830727e7297115ae, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:20.488157 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637221cb-bdbc-4401-b72e-a738b7fc1ebb admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:20.500981 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1401/2817] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 12:06:20 2022] PUT /identity/v3/projects/125fdf6902704f40afe5f94a3f3bfb80/users/0be6ec7b80324dac830727e7297115ae/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:06:20.511899 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471e311-5585-467d-a448-9aea103adecc None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:20.512202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471e311-5585-467d-a448-9aea103adecc None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:20.512483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471e311-5585-467d-a448-9aea103adecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:20.538165 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-0471e311-5585-467d-a448-9aea103adecc None None] MFA Rules not processed for user `0be6ec7b80324dac830727e7297115ae`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:06:20.539420 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-0471e311-5585-467d-a448-9aea103adecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:20.601739 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1417/2818] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:06:20 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 12:06:20.606145 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637221cb-bdbc-4401-b72e-a738b7fc1ebb admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:20.666196 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637221cb-bdbc-4401-b72e-a738b7fc1ebb admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:20.670841 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6dc5d-b46f-4a85-8d2f-9622c520abad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:20.671533 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6dc5d-b46f-4a85-8d2f-9622c520abad admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:20.671820 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6dc5d-b46f-4a85-8d2f-9622c520abad admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:20.672099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6dc5d-b46f-4a85-8d2f-9622c520abad admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:20.672788 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6dc5d-b46f-4a85-8d2f-9622c520abad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:20.674669 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6dc5d-b46f-4a85-8d2f-9622c520abad admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:20.699680 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1402/2819] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 12:06:20 2022] POST /identity/v3/projects => generated 365 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 12:06:20.703755 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471e311-5585-467d-a448-9aea103adecc None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:20.714352 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471e311-5585-467d-a448-9aea103adecc None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:20.768547 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8eb936-7561-407c-a389-22471d817a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:20.769202 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8eb936-7561-407c-a389-22471d817a92 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:20.769483 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8eb936-7561-407c-a389-22471d817a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:20.769754 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8eb936-7561-407c-a389-22471d817a92 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:20.770424 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8eb936-7561-407c-a389-22471d817a92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:20.772356 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8eb936-7561-407c-a389-22471d817a92 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:20.791236 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1418/2820] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 12:06:20 2022] POST /identity/v3/users => generated 289 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 12:06:20.795152 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6dc5d-b46f-4a85-8d2f-9622c520abad admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:20.805246 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6dc5d-b46f-4a85-8d2f-9622c520abad admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:20.810440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da004055-2d5f-4bb2-8d3a-da389ce7656d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:20.811097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da004055-2d5f-4bb2-8d3a-da389ce7656d admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:20.811414 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da004055-2d5f-4bb2-8d3a-da389ce7656d admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:20.811724 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da004055-2d5f-4bb2-8d3a-da389ce7656d admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:20.812443 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da004055-2d5f-4bb2-8d3a-da389ce7656d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:20.814044 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da004055-2d5f-4bb2-8d3a-da389ce7656d admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:20.824336 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1403/2821] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:06:20.828184 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8eb936-7561-407c-a389-22471d817a92 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:20.838191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8eb936-7561-407c-a389-22471d817a92 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:20.842765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d4171-4931-49d7-987c-b51114dc015a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:20.843664 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d4171-4931-49d7-987c-b51114dc015a admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:20.843947 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d4171-4931-49d7-987c-b51114dc015a admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:20.844223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d4171-4931-49d7-987c-b51114dc015a admin admin] PATH_INFO: `/v3/projects/6e364fceabd1451db7246e4b0025940f/users/9c1ee25197d943ecb77e1d38229c25b8/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:20.859101 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d4171-4931-49d7-987c-b51114dc015a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e364fceabd1451db7246e4b0025940f, user_id=9c1ee25197d943ecb77e1d38229c25b8, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:20.861749 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d4171-4931-49d7-987c-b51114dc015a admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:20.876018 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1419/2822] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 12:06:20 2022] PUT /identity/v3/projects/6e364fceabd1451db7246e4b0025940f/users/9c1ee25197d943ecb77e1d38229c25b8/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:06:20.879865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da004055-2d5f-4bb2-8d3a-da389ce7656d admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:20.889797 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da004055-2d5f-4bb2-8d3a-da389ce7656d admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:20.919710 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:06:20.950546 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d12ad9-7ef0-446c-ba62-e8707131efca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:20.951246 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d12ad9-7ef0-446c-ba62-e8707131efca admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:20.951645 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d12ad9-7ef0-446c-ba62-e8707131efca admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:20.951924 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d12ad9-7ef0-446c-ba62-e8707131efca admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:20.952641 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d12ad9-7ef0-446c-ba62-e8707131efca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:20.954215 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d12ad9-7ef0-446c-ba62-e8707131efca admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:20.963863 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1404/2823] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:06:20.968034 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d4171-4931-49d7-987c-b51114dc015a admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:20.978191 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d4171-4931-49d7-987c-b51114dc015a admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:20.983291 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f5e19-b41d-4a74-b1db-33bf2ddc1817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:20.984188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f5e19-b41d-4a74-b1db-33bf2ddc1817 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:20.984472 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f5e19-b41d-4a74-b1db-33bf2ddc1817 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:20.984748 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f5e19-b41d-4a74-b1db-33bf2ddc1817 admin admin] PATH_INFO: `/v3/projects/6e364fceabd1451db7246e4b0025940f/users/9c1ee25197d943ecb77e1d38229c25b8/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:20.986701 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f5e19-b41d-4a74-b1db-33bf2ddc1817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e364fceabd1451db7246e4b0025940f, user_id=9c1ee25197d943ecb77e1d38229c25b8, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:20.989360 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f5e19-b41d-4a74-b1db-33bf2ddc1817 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:21.000177 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1420/2824] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 12:06:20 2022] PUT /identity/v3/projects/6e364fceabd1451db7246e4b0025940f/users/9c1ee25197d943ecb77e1d38229c25b8/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:06:21.009966 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9a76a-02c4-4874-9d89-64839b889e19 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:21.010262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9a76a-02c4-4874-9d89-64839b889e19 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:21.010549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9a76a-02c4-4874-9d89-64839b889e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:21.036329 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-81b9a76a-02c4-4874-9d89-64839b889e19 None None] MFA Rules not processed for user `9c1ee25197d943ecb77e1d38229c25b8`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:06:21.037542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-81b9a76a-02c4-4874-9d89-64839b889e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:21.104589 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1405/2825] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:06:21 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:06:21.117340 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f5e19-b41d-4a74-b1db-33bf2ddc1817 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:21.119706 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e94f5e19-b41d-4a74-b1db-33bf2ddc1817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:21.161158 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:06:21.178146 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f5e19-b41d-4a74-b1db-33bf2ddc1817 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:21.181814 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445c948-4c04-4e06-b28d-cbf21dc280dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:21.182450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445c948-4c04-4e06-b28d-cbf21dc280dd service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:21.182740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445c948-4c04-4e06-b28d-cbf21dc280dd service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:21.183021 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445c948-4c04-4e06-b28d-cbf21dc280dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:21.191997 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445c948-4c04-4e06-b28d-cbf21dc280dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:21.193428 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445c948-4c04-4e06-b28d-cbf21dc280dd service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:21.202805 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1421/2826] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:06:21 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:06:21.271085 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9a76a-02c4-4874-9d89-64839b889e19 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:21.281435 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9a76a-02c4-4874-9d89-64839b889e19 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:21.285089 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55beef-ca71-40bc-823a-02ae6bfc9713 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:21.285723 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55beef-ca71-40bc-823a-02ae6bfc9713 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:21.286006 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55beef-ca71-40bc-823a-02ae6bfc9713 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:21.286283 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55beef-ca71-40bc-823a-02ae6bfc9713 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:21.345529 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55beef-ca71-40bc-823a-02ae6bfc9713 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:21.347011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55beef-ca71-40bc-823a-02ae6bfc9713 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:21.356776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1406/2827] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:06:21 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:06:24.035913 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445c948-4c04-4e06-b28d-cbf21dc280dd service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:24.097309 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445c948-4c04-4e06-b28d-cbf21dc280dd service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:24.101790 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc6e13-38c9-4223-8d1f-ecb8d724b9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:24.102642 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc6e13-38c9-4223-8d1f-ecb8d724b9bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:24.102930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc6e13-38c9-4223-8d1f-ecb8d724b9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:24.103779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc6e13-38c9-4223-8d1f-ecb8d724b9bd admin admin] PATH_INFO: `/v3/users/0be6ec7b80324dac830727e7297115ae` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:24.104955 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc6e13-38c9-4223-8d1f-ecb8d724b9bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0be6ec7b80324dac830727e7297115ae)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:24.106852 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc6e13-38c9-4223-8d1f-ecb8d724b9bd admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:24.151740 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-ddcc6e13-38c9-4223-8d1f-ecb8d724b9bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0be6ec7b80324dac830727e7297115ae'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:24.163615 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-ddcc6e13-38c9-4223-8d1f-ecb8d724b9bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0be6ec7b80324dac830727e7297115ae'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:24.171111 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-ddcc6e13-38c9-4223-8d1f-ecb8d724b9bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0be6ec7b80324dac830727e7297115ae'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:24.178780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-ddcc6e13-38c9-4223-8d1f-ecb8d724b9bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0be6ec7b80324dac830727e7297115ae'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:24.179516 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-ddcc6e13-38c9-4223-8d1f-ecb8d724b9bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0be6ec7b80324dac830727e7297115ae'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:24.185478 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1422/2828] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 12:06:24 2022] DELETE /identity/v3/users/0be6ec7b80324dac830727e7297115ae => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:06:24.189903 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731917a6-4e51-4666-8251-f37e09c13f96 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:24.190395 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731917a6-4e51-4666-8251-f37e09c13f96 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:24.190508 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731917a6-4e51-4666-8251-f37e09c13f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:24.207553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-731917a6-4e51-4666-8251-f37e09c13f96 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:06:24.208734 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-731917a6-4e51-4666-8251-f37e09c13f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:24.253090 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=100 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:06:24.267216 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1407/2829] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:06:24.277414 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc6e13-38c9-4223-8d1f-ecb8d724b9bd admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:24.278780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ddcc6e13-38c9-4223-8d1f-ecb8d724b9bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:24.335633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc6e13-38c9-4223-8d1f-ecb8d724b9bd admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:24.339915 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b448bd3-6965-428e-a8b6-80850a6beac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:24.340613 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b448bd3-6965-428e-a8b6-80850a6beac1 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:24.340893 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b448bd3-6965-428e-a8b6-80850a6beac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:24.341166 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b448bd3-6965-428e-a8b6-80850a6beac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:24.350226 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b448bd3-6965-428e-a8b6-80850a6beac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:24.351670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b448bd3-6965-428e-a8b6-80850a6beac1 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:24.361575 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1423/2830] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:06:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:06:24.619830 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731917a6-4e51-4666-8251-f37e09c13f96 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:24.621354 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-731917a6-4e51-4666-8251-f37e09c13f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:24.635745 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731917a6-4e51-4666-8251-f37e09c13f96 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:24.640440 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48191ee-4882-4148-9da5-a13c9f05da06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:24.641434 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48191ee-4882-4148-9da5-a13c9f05da06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:24.641715 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48191ee-4882-4148-9da5-a13c9f05da06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:24.642713 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48191ee-4882-4148-9da5-a13c9f05da06 admin admin] PATH_INFO: `/v3/projects/125fdf6902704f40afe5f94a3f3bfb80` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:24.644114 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48191ee-4882-4148-9da5-a13c9f05da06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=125fdf6902704f40afe5f94a3f3bfb80)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:24.646055 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48191ee-4882-4148-9da5-a13c9f05da06 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:24.691369 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b48191ee-4882-4148-9da5-a13c9f05da06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '125fdf6902704f40afe5f94a3f3bfb80', 'request_id': 'req-b48191ee-4882-4148-9da5-a13c9f05da06'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:24.699573 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b48191ee-4882-4148-9da5-a13c9f05da06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '125fdf6902704f40afe5f94a3f3bfb80', 'request_id': 'req-b48191ee-4882-4148-9da5-a13c9f05da06'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:24.705647 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b48191ee-4882-4148-9da5-a13c9f05da06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '125fdf6902704f40afe5f94a3f3bfb80', 'request_id': 'req-b48191ee-4882-4148-9da5-a13c9f05da06'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:24.716534 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b48191ee-4882-4148-9da5-a13c9f05da06 admin admin] The token cache is being invalidate because project 125fdf6902704f40afe5f94a3f3bfb80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 12:06:24.716823 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b48191ee-4882-4148-9da5-a13c9f05da06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:24.717454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-b48191ee-4882-4148-9da5-a13c9f05da06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:24.719038 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1408/2831] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 12:06:24 2022] DELETE /identity/v3/projects/125fdf6902704f40afe5f94a3f3bfb80 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:06:24.722952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b448bd3-6965-428e-a8b6-80850a6beac1 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:24.727911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:06:24.734044 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b448bd3-6965-428e-a8b6-80850a6beac1 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:24.789192 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68292e6f-94b3-4855-b367-29f2e71e4af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:24.790046 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68292e6f-94b3-4855-b367-29f2e71e4af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:24.790330 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68292e6f-94b3-4855-b367-29f2e71e4af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:24.790637 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68292e6f-94b3-4855-b367-29f2e71e4af9 admin admin] PATH_INFO: `/v3/users/9c1ee25197d943ecb77e1d38229c25b8` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:24.791912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68292e6f-94b3-4855-b367-29f2e71e4af9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c1ee25197d943ecb77e1d38229c25b8)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:24.793823 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68292e6f-94b3-4855-b367-29f2e71e4af9 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:24.835899 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-68292e6f-94b3-4855-b367-29f2e71e4af9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c1ee25197d943ecb77e1d38229c25b8'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:24.847921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-68292e6f-94b3-4855-b367-29f2e71e4af9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c1ee25197d943ecb77e1d38229c25b8'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:24.855139 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-68292e6f-94b3-4855-b367-29f2e71e4af9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c1ee25197d943ecb77e1d38229c25b8'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:24.862209 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-68292e6f-94b3-4855-b367-29f2e71e4af9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c1ee25197d943ecb77e1d38229c25b8'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:24.862839 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-68292e6f-94b3-4855-b367-29f2e71e4af9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c1ee25197d943ecb77e1d38229c25b8'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:24.868679 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1424/2832] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 12:06:24 2022] DELETE /identity/v3/users/9c1ee25197d943ecb77e1d38229c25b8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:06:25.132549 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48191ee-4882-4148-9da5-a13c9f05da06 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:25.133989 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b48191ee-4882-4148-9da5-a13c9f05da06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:25.147791 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48191ee-4882-4148-9da5-a13c9f05da06 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:25.203865 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499af4af-675f-4a7f-94ff-79c0975f6f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:25.204782 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499af4af-675f-4a7f-94ff-79c0975f6f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:25.205066 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499af4af-675f-4a7f-94ff-79c0975f6f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:25.205339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499af4af-675f-4a7f-94ff-79c0975f6f52 admin admin] PATH_INFO: `/v3/projects/6e364fceabd1451db7246e4b0025940f` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:25.206517 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499af4af-675f-4a7f-94ff-79c0975f6f52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e364fceabd1451db7246e4b0025940f)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:25.208455 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499af4af-675f-4a7f-94ff-79c0975f6f52 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:25.253564 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-499af4af-675f-4a7f-94ff-79c0975f6f52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e364fceabd1451db7246e4b0025940f', 'request_id': 'req-499af4af-675f-4a7f-94ff-79c0975f6f52'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:25.261090 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-499af4af-675f-4a7f-94ff-79c0975f6f52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e364fceabd1451db7246e4b0025940f', 'request_id': 'req-499af4af-675f-4a7f-94ff-79c0975f6f52'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:25.267126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-499af4af-675f-4a7f-94ff-79c0975f6f52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e364fceabd1451db7246e4b0025940f', 'request_id': 'req-499af4af-675f-4a7f-94ff-79c0975f6f52'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:25.277945 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-499af4af-675f-4a7f-94ff-79c0975f6f52 admin admin] The token cache is being invalidate because project 6e364fceabd1451db7246e4b0025940f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 12:06:25.278236 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-499af4af-675f-4a7f-94ff-79c0975f6f52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:25.278866 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-499af4af-675f-4a7f-94ff-79c0975f6f52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:06:25.280449 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1409/2833] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 12:06:25 2022] DELETE /identity/v3/projects/6e364fceabd1451db7246e4b0025940f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:06:25.285413 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1563af3-0001-4502-98ac-636b051f747e None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:25.285898 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1563af3-0001-4502-98ac-636b051f747e None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:25.286014 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1563af3-0001-4502-98ac-636b051f747e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:25.303714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-e1563af3-0001-4502-98ac-636b051f747e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:06:25.304899 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e1563af3-0001-4502-98ac-636b051f747e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:25.363007 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1425/2834] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:06:25.371328 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cac77-ffca-4302-bf3b-5d77613e45a9 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:25.371657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cac77-ffca-4302-bf3b-5d77613e45a9 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:25.371940 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cac77-ffca-4302-bf3b-5d77613e45a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:25.379102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:06:25.390406 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-012cac77-ffca-4302-bf3b-5d77613e45a9 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:06:25.392213 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-012cac77-ffca-4302-bf3b-5d77613e45a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:25.400385 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1410/2835] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:06:25.404590 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1563af3-0001-4502-98ac-636b051f747e None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:25.414230 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1563af3-0001-4502-98ac-636b051f747e None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:25.419299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0fced-0fc3-440a-9fda-0393ab311ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:25.419975 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0fced-0fc3-440a-9fda-0393ab311ddd admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:25.420265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0fced-0fc3-440a-9fda-0393ab311ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:25.420541 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0fced-0fc3-440a-9fda-0393ab311ddd admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:25.421260 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0fced-0fc3-440a-9fda-0393ab311ddd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 12:06:25.421568 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0fced-0fc3-440a-9fda-0393ab311ddd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:25.423168 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0fced-0fc3-440a-9fda-0393ab311ddd admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:25.435002 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1426/2836] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 12:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 12:06:25.439043 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012cac77-ffca-4302-bf3b-5d77613e45a9 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:25.448335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012cac77-ffca-4302-bf3b-5d77613e45a9 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:25.452955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63856eef-a2ab-4557-bda3-588dcc8b0cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:25.453595 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63856eef-a2ab-4557-bda3-588dcc8b0cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:25.453878 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63856eef-a2ab-4557-bda3-588dcc8b0cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:25.454149 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63856eef-a2ab-4557-bda3-588dcc8b0cd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:25.454892 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63856eef-a2ab-4557-bda3-588dcc8b0cd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:25.456801 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63856eef-a2ab-4557-bda3-588dcc8b0cd0 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:25.481860 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1411/2837] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 12:06:25 2022] POST /identity/v3/projects => generated 377 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 22 12:06:25.485801 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0fced-0fc3-440a-9fda-0393ab311ddd admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:25.496463 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0fced-0fc3-440a-9fda-0393ab311ddd admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:25.500954 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:06:25.555028 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49ba7a-1895-4f0e-aa63-a0111139f001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:25.555648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49ba7a-1895-4f0e-aa63-a0111139f001 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:25.555933 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49ba7a-1895-4f0e-aa63-a0111139f001 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:25.556238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49ba7a-1895-4f0e-aa63-a0111139f001 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:25.557466 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49ba7a-1895-4f0e-aa63-a0111139f001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:25.559438 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49ba7a-1895-4f0e-aa63-a0111139f001 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:25.578108 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1427/2838] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 12:06:25 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 12:06:25.582230 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63856eef-a2ab-4557-bda3-588dcc8b0cd0 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:25.592519 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63856eef-a2ab-4557-bda3-588dcc8b0cd0 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:25.597155 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a954f-ec85-4a0b-9813-d9517d661d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:25.597811 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a954f-ec85-4a0b-9813-d9517d661d22 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:25.598097 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a954f-ec85-4a0b-9813-d9517d661d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:25.598376 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a954f-ec85-4a0b-9813-d9517d661d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:25.599096 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a954f-ec85-4a0b-9813-d9517d661d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:25.600714 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a954f-ec85-4a0b-9813-d9517d661d22 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:25.611496 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1412/2839] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:06:25.615326 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49ba7a-1895-4f0e-aa63-a0111139f001 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:25.631449 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49ba7a-1895-4f0e-aa63-a0111139f001 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:25.637853 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1bffde-5560-48a8-a329-bc1b804098d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:25.638535 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bffde-5560-48a8-a329-bc1b804098d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:25.638815 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bffde-5560-48a8-a329-bc1b804098d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:25.639087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bffde-5560-48a8-a329-bc1b804098d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:25.639838 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1bffde-5560-48a8-a329-bc1b804098d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:25.641428 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1bffde-5560-48a8-a329-bc1b804098d8 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:25.652198 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1428/2840] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:06:25.656348 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a954f-ec85-4a0b-9813-d9517d661d22 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:25.666383 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a954f-ec85-4a0b-9813-d9517d661d22 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:25.671602 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d0ad9-8036-4f64-817b-9c59bbeae6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:25.672471 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d0ad9-8036-4f64-817b-9c59bbeae6c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:25.672753 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d0ad9-8036-4f64-817b-9c59bbeae6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:25.673030 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d0ad9-8036-4f64-817b-9c59bbeae6c5 admin admin] PATH_INFO: `/v3/projects/49aa7e81d147403d87ecf8090ed15120/users/f116b58a904a49b88e405c056dbe8a0b/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:25.687121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d0ad9-8036-4f64-817b-9c59bbeae6c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49aa7e81d147403d87ecf8090ed15120, user_id=f116b58a904a49b88e405c056dbe8a0b, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:25.689783 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d0ad9-8036-4f64-817b-9c59bbeae6c5 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:25.703624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1413/2841] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 12:06:25 2022] PUT /identity/v3/projects/49aa7e81d147403d87ecf8090ed15120/users/f116b58a904a49b88e405c056dbe8a0b/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:06:25.714146 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894aefa-8bd2-438b-9d9c-803837131dc4 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:25.714433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894aefa-8bd2-438b-9d9c-803837131dc4 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:25.714716 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f894aefa-8bd2-438b-9d9c-803837131dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:25.740912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f894aefa-8bd2-438b-9d9c-803837131dc4 None None] MFA Rules not processed for user `f116b58a904a49b88e405c056dbe8a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:06:25.742128 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f894aefa-8bd2-438b-9d9c-803837131dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:25.808006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1429/2842] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:06:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:06:25.812332 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d0ad9-8036-4f64-817b-9c59bbeae6c5 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:25.877966 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d0ad9-8036-4f64-817b-9c59bbeae6c5 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:25.882686 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa2ac2-66c4-4f1a-8bd7-fd44812f6800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:25.883371 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa2ac2-66c4-4f1a-8bd7-fd44812f6800 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:25.883649 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa2ac2-66c4-4f1a-8bd7-fd44812f6800 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:25.883931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa2ac2-66c4-4f1a-8bd7-fd44812f6800 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:25.885174 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa2ac2-66c4-4f1a-8bd7-fd44812f6800 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:25.887067 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa2ac2-66c4-4f1a-8bd7-fd44812f6800 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:25.912027 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1414/2843] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 12:06:25 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 12:06:25.916307 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894aefa-8bd2-438b-9d9c-803837131dc4 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:25.927012 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f894aefa-8bd2-438b-9d9c-803837131dc4 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:25.982608 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af966521-2ee9-419b-ab5d-7079aabdadd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:25.983268 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af966521-2ee9-419b-ab5d-7079aabdadd4 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:25.983576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af966521-2ee9-419b-ab5d-7079aabdadd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:25.983852 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af966521-2ee9-419b-ab5d-7079aabdadd4 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:25.984539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af966521-2ee9-419b-ab5d-7079aabdadd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:25.986441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af966521-2ee9-419b-ab5d-7079aabdadd4 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:26.006101 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1430/2844] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 12:06:25 2022] POST /identity/v3/users => generated 295 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 12:06:26.010870 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa2ac2-66c4-4f1a-8bd7-fd44812f6800 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:26.021085 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa2ac2-66c4-4f1a-8bd7-fd44812f6800 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:26.026003 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93e034-c309-42a6-b46c-43fddd9e8b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:26.026659 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93e034-c309-42a6-b46c-43fddd9e8b0b admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:26.026950 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93e034-c309-42a6-b46c-43fddd9e8b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:26.027230 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93e034-c309-42a6-b46c-43fddd9e8b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:26.027990 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93e034-c309-42a6-b46c-43fddd9e8b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:26.029580 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93e034-c309-42a6-b46c-43fddd9e8b0b admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:26.039876 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1415/2845] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:06:26.044238 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af966521-2ee9-419b-ab5d-7079aabdadd4 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:26.054156 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af966521-2ee9-419b-ab5d-7079aabdadd4 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:26.059538 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf427734-807f-490b-b730-5581f27253a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:26.060350 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf427734-807f-490b-b730-5581f27253a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:26.060633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf427734-807f-490b-b730-5581f27253a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:26.060909 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf427734-807f-490b-b730-5581f27253a0 admin admin] PATH_INFO: `/v3/projects/b16200a1464042cda4e5b32fd5d25a48/users/369b41db86844507bab40382ef8f05da/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:26.063109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:06:26.077243 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf427734-807f-490b-b730-5581f27253a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16200a1464042cda4e5b32fd5d25a48, user_id=369b41db86844507bab40382ef8f05da, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:26.079971 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf427734-807f-490b-b730-5581f27253a0 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:26.092353 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1431/2846] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 12:06:26 2022] PUT /identity/v3/projects/b16200a1464042cda4e5b32fd5d25a48/users/369b41db86844507bab40382ef8f05da/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:06:26.096339 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93e034-c309-42a6-b46c-43fddd9e8b0b admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:26.106933 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93e034-c309-42a6-b46c-43fddd9e8b0b admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:26.161394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f700ae-c591-4761-82c0-713dfc1ce7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:26.162046 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f700ae-c591-4761-82c0-713dfc1ce7b0 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:26.162327 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f700ae-c591-4761-82c0-713dfc1ce7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:26.162605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f700ae-c591-4761-82c0-713dfc1ce7b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:26.163320 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f700ae-c591-4761-82c0-713dfc1ce7b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:26.164917 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f700ae-c591-4761-82c0-713dfc1ce7b0 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:26.174892 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1416/2847] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:06:26.178911 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf427734-807f-490b-b730-5581f27253a0 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:26.189843 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf427734-807f-490b-b730-5581f27253a0 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:26.194462 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee875275-1eff-40fb-bc1b-96bc41649e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:26.195379 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee875275-1eff-40fb-bc1b-96bc41649e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:26.195676 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee875275-1eff-40fb-bc1b-96bc41649e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:26.195956 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee875275-1eff-40fb-bc1b-96bc41649e7e admin admin] PATH_INFO: `/v3/projects/b16200a1464042cda4e5b32fd5d25a48/users/369b41db86844507bab40382ef8f05da/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:26.197912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee875275-1eff-40fb-bc1b-96bc41649e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16200a1464042cda4e5b32fd5d25a48, user_id=369b41db86844507bab40382ef8f05da, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:26.200594 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee875275-1eff-40fb-bc1b-96bc41649e7e admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:26.210699 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1432/2848] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 12:06:26 2022] PUT /identity/v3/projects/b16200a1464042cda4e5b32fd5d25a48/users/369b41db86844507bab40382ef8f05da/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:06:26.272394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fec257-d5fa-487c-aa9a-ccbab3a33dc0 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:26.272979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fec257-d5fa-487c-aa9a-ccbab3a33dc0 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:26.272979 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fec257-d5fa-487c-aa9a-ccbab3a33dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:26.292805 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:06:26.300525 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-71fec257-d5fa-487c-aa9a-ccbab3a33dc0 None None] MFA Rules not processed for user `369b41db86844507bab40382ef8f05da`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:06:26.302468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-71fec257-d5fa-487c-aa9a-ccbab3a33dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:26.368319 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1417/2849] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:06:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:06:26.379274 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee875275-1eff-40fb-bc1b-96bc41649e7e admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:26.381037 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ee875275-1eff-40fb-bc1b-96bc41649e7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:26.438762 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee875275-1eff-40fb-bc1b-96bc41649e7e admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:26.442480 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0e350-e661-4d80-950c-0dd8e8e2fd41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:26.443120 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0e350-e661-4d80-950c-0dd8e8e2fd41 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:26.443443 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0e350-e661-4d80-950c-0dd8e8e2fd41 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:26.443744 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0e350-e661-4d80-950c-0dd8e8e2fd41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:26.498298 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0e350-e661-4d80-950c-0dd8e8e2fd41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:26.499758 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0e350-e661-4d80-950c-0dd8e8e2fd41 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:26.508496 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1433/2850] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:06:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:06:32.112101 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fec257-d5fa-487c-aa9a-ccbab3a33dc0 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:32.113498 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-71fec257-d5fa-487c-aa9a-ccbab3a33dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:06:32.126078 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fec257-d5fa-487c-aa9a-ccbab3a33dc0 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:32.129724 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91108476-f403-4d11-b17a-3305867c66ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:32.130281 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91108476-f403-4d11-b17a-3305867c66ac service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:32.130569 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91108476-f403-4d11-b17a-3305867c66ac service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:32.130845 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91108476-f403-4d11-b17a-3305867c66ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:32.139775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91108476-f403-4d11-b17a-3305867c66ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:32.141189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91108476-f403-4d11-b17a-3305867c66ac service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:06:32.150855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1418/2851] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:06:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:06:36.504508 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=60,keystone:INSERT=2 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:06:42.146368 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=35 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:06:59.911921 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0e350-e661-4d80-950c-0dd8e8e2fd41 service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:06:59.973450 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0e350-e661-4d80-950c-0dd8e8e2fd41 service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:06:59.978008 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e65c05-bfb1-454c-9f34-37e66a4e1efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:06:59.978864 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e65c05-bfb1-454c-9f34-37e66a4e1efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:06:59.979149 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e65c05-bfb1-454c-9f34-37e66a4e1efd admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:06:59.979980 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e65c05-bfb1-454c-9f34-37e66a4e1efd admin admin] PATH_INFO: `/v3/users/f116b58a904a49b88e405c056dbe8a0b` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:06:59.981163 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e65c05-bfb1-454c-9f34-37e66a4e1efd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f116b58a904a49b88e405c056dbe8a0b)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:06:59.983073 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e65c05-bfb1-454c-9f34-37e66a4e1efd admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:00.028405 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b8e65c05-bfb1-454c-9f34-37e66a4e1efd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f116b58a904a49b88e405c056dbe8a0b'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:00.040086 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b8e65c05-bfb1-454c-9f34-37e66a4e1efd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f116b58a904a49b88e405c056dbe8a0b'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:00.048426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b8e65c05-bfb1-454c-9f34-37e66a4e1efd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f116b58a904a49b88e405c056dbe8a0b'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:00.055205 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b8e65c05-bfb1-454c-9f34-37e66a4e1efd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f116b58a904a49b88e405c056dbe8a0b'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:00.055849 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-b8e65c05-bfb1-454c-9f34-37e66a4e1efd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f116b58a904a49b88e405c056dbe8a0b'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:00.062356 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1434/2852] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 12:06:59 2022] DELETE /identity/v3/users/f116b58a904a49b88e405c056dbe8a0b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:07:00.066802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f0ca4-49c5-444b-81c0-adb18f79a240 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:00.067307 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f0ca4-49c5-444b-81c0-adb18f79a240 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:00.067454 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f0ca4-49c5-444b-81c0-adb18f79a240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:00.085121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-6c3f0ca4-49c5-444b-81c0-adb18f79a240 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:07:00.086325 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-6c3f0ca4-49c5-444b-81c0-adb18f79a240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:00.143716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1419/2853] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:07:00.153966 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e65c05-bfb1-454c-9f34-37e66a4e1efd admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:00.155367 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-b8e65c05-bfb1-454c-9f34-37e66a4e1efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:00.212225 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e65c05-bfb1-454c-9f34-37e66a4e1efd admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:00.216469 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96599939-f4f1-45f6-a6a1-41378876e549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:00.217110 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96599939-f4f1-45f6-a6a1-41378876e549 service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:00.217392 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96599939-f4f1-45f6-a6a1-41378876e549 service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:00.217670 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96599939-f4f1-45f6-a6a1-41378876e549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:00.226512 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96599939-f4f1-45f6-a6a1-41378876e549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:00.227958 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96599939-f4f1-45f6-a6a1-41378876e549 service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:00.237825 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1435/2854] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:07:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:07:00.496875 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f0ca4-49c5-444b-81c0-adb18f79a240 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:00.498289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-6c3f0ca4-49c5-444b-81c0-adb18f79a240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:00.510205 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f0ca4-49c5-444b-81c0-adb18f79a240 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:00.515716 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa48ac6-6da9-4bad-96db-b131cd821d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:00.516634 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa48ac6-6da9-4bad-96db-b131cd821d4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:00.516926 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa48ac6-6da9-4bad-96db-b131cd821d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:00.517209 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa48ac6-6da9-4bad-96db-b131cd821d4c admin admin] PATH_INFO: `/v3/projects/49aa7e81d147403d87ecf8090ed15120` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:00.518415 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa48ac6-6da9-4bad-96db-b131cd821d4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=49aa7e81d147403d87ecf8090ed15120)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:00.520522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa48ac6-6da9-4bad-96db-b131cd821d4c admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:00.566784 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1aa48ac6-6da9-4bad-96db-b131cd821d4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49aa7e81d147403d87ecf8090ed15120', 'request_id': 'req-1aa48ac6-6da9-4bad-96db-b131cd821d4c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:00.574254 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1aa48ac6-6da9-4bad-96db-b131cd821d4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49aa7e81d147403d87ecf8090ed15120', 'request_id': 'req-1aa48ac6-6da9-4bad-96db-b131cd821d4c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:00.580451 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1aa48ac6-6da9-4bad-96db-b131cd821d4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49aa7e81d147403d87ecf8090ed15120', 'request_id': 'req-1aa48ac6-6da9-4bad-96db-b131cd821d4c'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:00.591296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1aa48ac6-6da9-4bad-96db-b131cd821d4c admin admin] The token cache is being invalidate because project 49aa7e81d147403d87ecf8090ed15120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 12:07:00.591610 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1aa48ac6-6da9-4bad-96db-b131cd821d4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:00.592303 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-1aa48ac6-6da9-4bad-96db-b131cd821d4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:00.593827 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1420/2855] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 12:07:00 2022] DELETE /identity/v3/projects/49aa7e81d147403d87ecf8090ed15120 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:07:00.597756 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96599939-f4f1-45f6-a6a1-41378876e549 service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:00.607170 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96599939-f4f1-45f6-a6a1-41378876e549 service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:00.624851 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:07:00.665279 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74f650-6f43-438e-b003-42da4e7345d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:00.666142 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74f650-6f43-438e-b003-42da4e7345d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:00.666429 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74f650-6f43-438e-b003-42da4e7345d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:00.666711 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74f650-6f43-438e-b003-42da4e7345d5 admin admin] PATH_INFO: `/v3/users/369b41db86844507bab40382ef8f05da` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:00.668080 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74f650-6f43-438e-b003-42da4e7345d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=369b41db86844507bab40382ef8f05da)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:00.670011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74f650-6f43-438e-b003-42da4e7345d5 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:00.716616 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-cf74f650-6f43-438e-b003-42da4e7345d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '369b41db86844507bab40382ef8f05da'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:00.728348 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-cf74f650-6f43-438e-b003-42da4e7345d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '369b41db86844507bab40382ef8f05da'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:00.735897 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-cf74f650-6f43-438e-b003-42da4e7345d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '369b41db86844507bab40382ef8f05da'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:00.742979 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-cf74f650-6f43-438e-b003-42da4e7345d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '369b41db86844507bab40382ef8f05da'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:00.743633 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-cf74f650-6f43-438e-b003-42da4e7345d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '369b41db86844507bab40382ef8f05da'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:00.749512 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1436/2856] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 12:07:00 2022] DELETE /identity/v3/users/369b41db86844507bab40382ef8f05da => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:07:01.013175 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa48ac6-6da9-4bad-96db-b131cd821d4c admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:01.014718 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-1aa48ac6-6da9-4bad-96db-b131cd821d4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:01.030993 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa48ac6-6da9-4bad-96db-b131cd821d4c admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:01.088537 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745534ab-0519-4f69-a4ee-758f1cea5b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:01.089911 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745534ab-0519-4f69-a4ee-758f1cea5b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:01.090198 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745534ab-0519-4f69-a4ee-758f1cea5b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:01.090478 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745534ab-0519-4f69-a4ee-758f1cea5b6a admin admin] PATH_INFO: `/v3/projects/b16200a1464042cda4e5b32fd5d25a48` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:01.091705 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745534ab-0519-4f69-a4ee-758f1cea5b6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b16200a1464042cda4e5b32fd5d25a48)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:01.093604 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745534ab-0519-4f69-a4ee-758f1cea5b6a admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:01.123584 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:07:01.140470 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-745534ab-0519-4f69-a4ee-758f1cea5b6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b16200a1464042cda4e5b32fd5d25a48', 'request_id': 'req-745534ab-0519-4f69-a4ee-758f1cea5b6a'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:01.147785 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-745534ab-0519-4f69-a4ee-758f1cea5b6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b16200a1464042cda4e5b32fd5d25a48', 'request_id': 'req-745534ab-0519-4f69-a4ee-758f1cea5b6a'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:01.154758 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-745534ab-0519-4f69-a4ee-758f1cea5b6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b16200a1464042cda4e5b32fd5d25a48', 'request_id': 'req-745534ab-0519-4f69-a4ee-758f1cea5b6a'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:01.165394 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-745534ab-0519-4f69-a4ee-758f1cea5b6a admin admin] The token cache is being invalidate because project b16200a1464042cda4e5b32fd5d25a48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86768) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 12:07:01.165687 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-745534ab-0519-4f69-a4ee-758f1cea5b6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:01.166340 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-745534ab-0519-4f69-a4ee-758f1cea5b6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:01.168495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1421/2857] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 12:07:01 2022] DELETE /identity/v3/projects/b16200a1464042cda4e5b32fd5d25a48 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:07:01.173618 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab3b22-578d-4a35-b29f-bcbd16663db5 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:01.174094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab3b22-578d-4a35-b29f-bcbd16663db5 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:01.174205 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab3b22-578d-4a35-b29f-bcbd16663db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:01.191784 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-24ab3b22-578d-4a35-b29f-bcbd16663db5 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:07:01.192999 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-24ab3b22-578d-4a35-b29f-bcbd16663db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:01.251464 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1437/2858] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:07:01.259003 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a1edf-b334-45a6-b5f1-6da892d4c872 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:01.259242 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a1edf-b334-45a6-b5f1-6da892d4c872 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:01.259556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a1edf-b334-45a6-b5f1-6da892d4c872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:01.278689 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-cb8a1edf-b334-45a6-b5f1-6da892d4c872 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:07:01.279926 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-cb8a1edf-b334-45a6-b5f1-6da892d4c872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:01.287900 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1422/2859] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:07:01.292137 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab3b22-578d-4a35-b29f-bcbd16663db5 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:01.301690 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab3b22-578d-4a35-b29f-bcbd16663db5 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:01.306222 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e8432-0c0a-4fd3-9216-12fd422a3ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:01.306872 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e8432-0c0a-4fd3-9216-12fd422a3ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:01.307158 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e8432-0c0a-4fd3-9216-12fd422a3ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:01.307474 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e8432-0c0a-4fd3-9216-12fd422a3ce8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:01.308193 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e8432-0c0a-4fd3-9216-12fd422a3ce8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 12:07:01.308499 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e8432-0c0a-4fd3-9216-12fd422a3ce8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:01.310089 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e8432-0c0a-4fd3-9216-12fd422a3ce8 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:01.324474 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1438/2860] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 12:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 12:07:01.329639 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a1edf-b334-45a6-b5f1-6da892d4c872 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:01.340020 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a1edf-b334-45a6-b5f1-6da892d4c872 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:01.344776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834ae7d-9479-40ba-80f1-9a738d8d06fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:01.345468 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834ae7d-9479-40ba-80f1-9a738d8d06fd admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:01.345727 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834ae7d-9479-40ba-80f1-9a738d8d06fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:01.346008 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834ae7d-9479-40ba-80f1-9a738d8d06fd admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:01.346698 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0834ae7d-9479-40ba-80f1-9a738d8d06fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:01.348643 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0834ae7d-9479-40ba-80f1-9a738d8d06fd admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:01.373693 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1423/2861] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 12:07:01 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 12:07:01.378090 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e8432-0c0a-4fd3-9216-12fd422a3ce8 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:01.388606 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e8432-0c0a-4fd3-9216-12fd422a3ce8 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:01.414423 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:07:01.444392 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8793f-0a18-4538-ac3f-d3942dd8f0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:01.445150 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8793f-0a18-4538-ac3f-d3942dd8f0e8 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:01.445523 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8793f-0a18-4538-ac3f-d3942dd8f0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:01.445801 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8793f-0a18-4538-ac3f-d3942dd8f0e8 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:01.446485 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8793f-0a18-4538-ac3f-d3942dd8f0e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:01.448925 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8793f-0a18-4538-ac3f-d3942dd8f0e8 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:01.469062 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1439/2862] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 12:07:01 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 12:07:01.473409 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834ae7d-9479-40ba-80f1-9a738d8d06fd admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:01.483753 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834ae7d-9479-40ba-80f1-9a738d8d06fd admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:01.488289 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73039654-12f4-433e-be5b-6b5c366c08c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:01.488942 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73039654-12f4-433e-be5b-6b5c366c08c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:01.489220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73039654-12f4-433e-be5b-6b5c366c08c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:01.489494 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73039654-12f4-433e-be5b-6b5c366c08c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:01.490210 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73039654-12f4-433e-be5b-6b5c366c08c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:01.491821 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73039654-12f4-433e-be5b-6b5c366c08c9 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:01.501989 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1424/2863] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:07:01.505788 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8793f-0a18-4538-ac3f-d3942dd8f0e8 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:01.516152 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8793f-0a18-4538-ac3f-d3942dd8f0e8 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:01.520668 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59e316-538f-45c4-ab9d-546d617381ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:01.521315 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59e316-538f-45c4-ab9d-546d617381ae admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:01.521604 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59e316-538f-45c4-ab9d-546d617381ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:01.521887 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59e316-538f-45c4-ab9d-546d617381ae admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:01.522608 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59e316-538f-45c4-ab9d-546d617381ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:01.524492 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59e316-538f-45c4-ab9d-546d617381ae admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:01.534807 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1440/2864] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:07:01.538805 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73039654-12f4-433e-be5b-6b5c366c08c9 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:01.548498 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73039654-12f4-433e-be5b-6b5c366c08c9 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:01.553122 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e210c-24e8-486f-a505-0dc19b2fc59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:01.554007 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e210c-24e8-486f-a505-0dc19b2fc59a admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:01.554282 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e210c-24e8-486f-a505-0dc19b2fc59a admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:01.554553 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e210c-24e8-486f-a505-0dc19b2fc59a admin admin] PATH_INFO: `/v3/projects/da61641e14e84854994796c6edf68b47/users/f6b469c6eaa14a5bb3120004f2870679/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:01.569609 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e210c-24e8-486f-a505-0dc19b2fc59a admin admin] RBAC: Authorizing `identity:create_grant(project_id=da61641e14e84854994796c6edf68b47, user_id=f6b469c6eaa14a5bb3120004f2870679, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:01.572239 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e210c-24e8-486f-a505-0dc19b2fc59a admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:01.585671 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1425/2865] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 12:07:01 2022] PUT /identity/v3/projects/da61641e14e84854994796c6edf68b47/users/f6b469c6eaa14a5bb3120004f2870679/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:07:01.597345 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd806d24-4752-4142-9bd9-c64195c335d1 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:01.597691 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd806d24-4752-4142-9bd9-c64195c335d1 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:01.598002 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd806d24-4752-4142-9bd9-c64195c335d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:01.624161 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-dd806d24-4752-4142-9bd9-c64195c335d1 None None] MFA Rules not processed for user `f6b469c6eaa14a5bb3120004f2870679`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:07:01.625390 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-dd806d24-4752-4142-9bd9-c64195c335d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:01.688631 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1441/2866] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:07:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:07:01.693184 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e210c-24e8-486f-a505-0dc19b2fc59a admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:01.752860 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e210c-24e8-486f-a505-0dc19b2fc59a admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:01.757584 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d913bfe-eb50-447b-a1aa-9401b266c589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:01.758237 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d913bfe-eb50-447b-a1aa-9401b266c589 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:01.758524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d913bfe-eb50-447b-a1aa-9401b266c589 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:01.758807 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d913bfe-eb50-447b-a1aa-9401b266c589 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:01.759518 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d913bfe-eb50-447b-a1aa-9401b266c589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:01.761970 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d913bfe-eb50-447b-a1aa-9401b266c589 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:01.788240 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1426/2867] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 12:07:01 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 12:07:01.792245 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd806d24-4752-4142-9bd9-c64195c335d1 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:01.802835 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd806d24-4752-4142-9bd9-c64195c335d1 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:01.857846 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7c673-4eba-4baf-93d0-0c92d0650598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:01.858502 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7c673-4eba-4baf-93d0-0c92d0650598 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:01.858784 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7c673-4eba-4baf-93d0-0c92d0650598 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:01.859060 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7c673-4eba-4baf-93d0-0c92d0650598 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:01.859779 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7c673-4eba-4baf-93d0-0c92d0650598 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:01.861680 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7c673-4eba-4baf-93d0-0c92d0650598 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:01.884011 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1442/2868] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 12:07:01 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 12:07:01.888364 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d913bfe-eb50-447b-a1aa-9401b266c589 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:01.898720 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d913bfe-eb50-447b-a1aa-9401b266c589 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:01.904866 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe293826-e670-4aab-a5bb-ae5e42da99ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:01.905662 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe293826-e670-4aab-a5bb-ae5e42da99ca admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:01.905955 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe293826-e670-4aab-a5bb-ae5e42da99ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:01.907321 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe293826-e670-4aab-a5bb-ae5e42da99ca admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:01.908142 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe293826-e670-4aab-a5bb-ae5e42da99ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:01.911090 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe293826-e670-4aab-a5bb-ae5e42da99ca admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:01.926330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1427/2869] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:07:01.930335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7c673-4eba-4baf-93d0-0c92d0650598 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:01.941097 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7c673-4eba-4baf-93d0-0c92d0650598 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:01.945834 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb6886-1267-466a-9191-6eb44b105edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:01.946484 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb6886-1267-466a-9191-6eb44b105edf admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:01.946765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb6886-1267-466a-9191-6eb44b105edf admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:01.947040 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb6886-1267-466a-9191-6eb44b105edf admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:01.947814 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb6886-1267-466a-9191-6eb44b105edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:01.950002 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb6886-1267-466a-9191-6eb44b105edf admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:01.960402 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1443/2870] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:07:01.964585 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe293826-e670-4aab-a5bb-ae5e42da99ca admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:01.974475 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe293826-e670-4aab-a5bb-ae5e42da99ca admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:01.979095 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46122f3-d1cf-4082-8e47-6fd281bc702d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:01.980511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46122f3-d1cf-4082-8e47-6fd281bc702d admin admin] REQUEST_METHOD: `PUT` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:01.980795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46122f3-d1cf-4082-8e47-6fd281bc702d admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:01.981093 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46122f3-d1cf-4082-8e47-6fd281bc702d admin admin] PATH_INFO: `/v3/projects/a2da056196544268a6ab5cad08834fac/users/0b88e3294e2141d8b28fca4a1ad73e4e/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:01.996162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46122f3-d1cf-4082-8e47-6fd281bc702d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2da056196544268a6ab5cad08834fac, user_id=0b88e3294e2141d8b28fca4a1ad73e4e, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:01.998775 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46122f3-d1cf-4082-8e47-6fd281bc702d admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:02.006810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:07:02.012285 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1428/2871] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 12:07:01 2022] PUT /identity/v3/projects/a2da056196544268a6ab5cad08834fac/users/0b88e3294e2141d8b28fca4a1ad73e4e/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:07:02.022726 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab691ff-63c9-40c2-ad78-e24401fbe95c None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:02.023020 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab691ff-63c9-40c2-ad78-e24401fbe95c None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:02.023299 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab691ff-63c9-40c2-ad78-e24401fbe95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:02.049442 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-4ab691ff-63c9-40c2-ad78-e24401fbe95c None None] MFA Rules not processed for user `0b88e3294e2141d8b28fca4a1ad73e4e`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:07:02.050566 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-4ab691ff-63c9-40c2-ad78-e24401fbe95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:02.054582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:07:02.117404 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1444/2872] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:07:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:07:02.122636 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46122f3-d1cf-4082-8e47-6fd281bc702d admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:02.183156 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46122f3-d1cf-4082-8e47-6fd281bc702d admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:02.187787 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d741b-8724-4d63-9381-a3c9b44f26b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:02.188436 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d741b-8724-4d63-9381-a3c9b44f26b3 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:02.188717 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d741b-8724-4d63-9381-a3c9b44f26b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:02.188990 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d741b-8724-4d63-9381-a3c9b44f26b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:02.189675 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d741b-8724-4d63-9381-a3c9b44f26b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:02.191605 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d741b-8724-4d63-9381-a3c9b44f26b3 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:02.216294 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1429/2873] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 12:07:02 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 12:07:02.220319 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab691ff-63c9-40c2-ad78-e24401fbe95c None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:02.231454 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab691ff-63c9-40c2-ad78-e24401fbe95c None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:02.292480 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb0b71-34ce-484e-891a-227898b2db11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:02.293148 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb0b71-34ce-484e-891a-227898b2db11 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:02.293435 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb0b71-34ce-484e-891a-227898b2db11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:02.293708 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb0b71-34ce-484e-891a-227898b2db11 admin admin] PATH_INFO: `/v3/users` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:02.294403 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb0b71-34ce-484e-891a-227898b2db11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:02.296353 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb0b71-34ce-484e-891a-227898b2db11 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:02.316052 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1445/2874] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 12:07:02 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 12:07:02.319958 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d741b-8724-4d63-9381-a3c9b44f26b3 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:02.330095 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d741b-8724-4d63-9381-a3c9b44f26b3 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:02.334757 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea72bfc-09ca-4e80-a9e3-54a134ea8dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:02.335467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea72bfc-09ca-4e80-a9e3-54a134ea8dfb admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:02.335754 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea72bfc-09ca-4e80-a9e3-54a134ea8dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:02.336029 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea72bfc-09ca-4e80-a9e3-54a134ea8dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:02.336748 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea72bfc-09ca-4e80-a9e3-54a134ea8dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:02.338330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea72bfc-09ca-4e80-a9e3-54a134ea8dfb admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:02.349173 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1430/2875] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:07:02.353075 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb0b71-34ce-484e-891a-227898b2db11 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:02.363710 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb0b71-34ce-484e-891a-227898b2db11 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:02.368301 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c23792-06ae-4fb9-ba08-e19c5051a5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:02.369183 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c23792-06ae-4fb9-ba08-e19c5051a5f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:02.369465 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c23792-06ae-4fb9-ba08-e19c5051a5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:02.369852 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c23792-06ae-4fb9-ba08-e19c5051a5f9 admin admin] PATH_INFO: `/v3/projects/6e9a6d4f5c944e80891b69156cd31790/users/0d0d4b15032741d9a82a59836cc208af/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:02.385648 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c23792-06ae-4fb9-ba08-e19c5051a5f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e9a6d4f5c944e80891b69156cd31790, user_id=0d0d4b15032741d9a82a59836cc208af, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:02.388411 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c23792-06ae-4fb9-ba08-e19c5051a5f9 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:02.401546 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1446/2876] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 12:07:02 2022] PUT /identity/v3/projects/6e9a6d4f5c944e80891b69156cd31790/users/0d0d4b15032741d9a82a59836cc208af/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:07:02.405363 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea72bfc-09ca-4e80-a9e3-54a134ea8dfb admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:02.415329 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea72bfc-09ca-4e80-a9e3-54a134ea8dfb admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:02.470888 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d258f8-8d9b-4e9b-8014-a4e452adf6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:02.471575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d258f8-8d9b-4e9b-8014-a4e452adf6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:02.471861 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d258f8-8d9b-4e9b-8014-a4e452adf6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:02.472139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d258f8-8d9b-4e9b-8014-a4e452adf6d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:02.472856 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d258f8-8d9b-4e9b-8014-a4e452adf6d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:02.474446 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d258f8-8d9b-4e9b-8014-a4e452adf6d7 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:02.484542 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1431/2877] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:07:02 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:07:02.488691 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c23792-06ae-4fb9-ba08-e19c5051a5f9 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:02.498770 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c23792-06ae-4fb9-ba08-e19c5051a5f9 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:02.503285 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d09ab6-0159-44bb-8235-94c3c3c693ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:02.504178 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d09ab6-0159-44bb-8235-94c3c3c693ff admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:02.504456 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d09ab6-0159-44bb-8235-94c3c3c693ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:02.504729 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d09ab6-0159-44bb-8235-94c3c3c693ff admin admin] PATH_INFO: `/v3/projects/6e9a6d4f5c944e80891b69156cd31790/users/0d0d4b15032741d9a82a59836cc208af/roles/96c584428d924fdba9671949f760c406` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:02.506660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d09ab6-0159-44bb-8235-94c3c3c693ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e9a6d4f5c944e80891b69156cd31790, user_id=0d0d4b15032741d9a82a59836cc208af, role_id=96c584428d924fdba9671949f760c406)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:02.509315 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d09ab6-0159-44bb-8235-94c3c3c693ff admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:02.519138 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1447/2878] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 12:07:02 2022] PUT /identity/v3/projects/6e9a6d4f5c944e80891b69156cd31790/users/0d0d4b15032741d9a82a59836cc208af/roles/96c584428d924fdba9671949f760c406 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:07:02.529511 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc967e12-2e05-4804-9526-4572513ca4b3 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:02.529802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc967e12-2e05-4804-9526-4572513ca4b3 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:02.530080 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc967e12-2e05-4804-9526-4572513ca4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:02.559318 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-cc967e12-2e05-4804-9526-4572513ca4b3 None None] MFA Rules not processed for user `0d0d4b15032741d9a82a59836cc208af`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:07:02.560624 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-cc967e12-2e05-4804-9526-4572513ca4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:02.619291 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:07:02.628818 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1432/2879] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:07:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:07:02.639885 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d09ab6-0159-44bb-8235-94c3c3c693ff admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:02.641569 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-85d09ab6-0159-44bb-8235-94c3c3c693ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:02.697640 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d09ab6-0159-44bb-8235-94c3c3c693ff admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:02.701265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab799c-ae86-4150-b7a6-019900cf1835 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:02.701888 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab799c-ae86-4150-b7a6-019900cf1835 service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:02.702168 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab799c-ae86-4150-b7a6-019900cf1835 service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:02.702441 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab799c-ae86-4150-b7a6-019900cf1835 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:02.711414 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:07:02.758121 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab799c-ae86-4150-b7a6-019900cf1835 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:02.759562 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab799c-ae86-4150-b7a6-019900cf1835 service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:02.768695 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1448/2880] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:07:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:07:04.244219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc967e12-2e05-4804-9526-4572513ca4b3 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:04.245737 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-cc967e12-2e05-4804-9526-4572513ca4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:04.259421 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc967e12-2e05-4804-9526-4572513ca4b3 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:04.263608 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b47acc-b2ad-4e4e-8e78-88fd384901d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:04.264251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b47acc-b2ad-4e4e-8e78-88fd384901d2 service cinder] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:04.264539 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b47acc-b2ad-4e4e-8e78-88fd384901d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:04.264819 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b47acc-b2ad-4e4e-8e78-88fd384901d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:04.321133 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b47acc-b2ad-4e4e-8e78-88fd384901d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:04.322562 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b47acc-b2ad-4e4e-8e78-88fd384901d2 service cinder] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:04.331949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1433/2881] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:07:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:07:04.832405 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab799c-ae86-4150-b7a6-019900cf1835 service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:04.841666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab799c-ae86-4150-b7a6-019900cf1835 service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:04.845377 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7c178-aab9-427e-9017-151840346a6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:04.846010 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7c178-aab9-427e-9017-151840346a6e service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:04.846297 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7c178-aab9-427e-9017-151840346a6e service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:04.846581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7c178-aab9-427e-9017-151840346a6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:04.856158 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7c178-aab9-427e-9017-151840346a6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:04.857579 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7c178-aab9-427e-9017-151840346a6e service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:04.867048 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1449/2882] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:07:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:07:08.666867 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b47acc-b2ad-4e4e-8e78-88fd384901d2 service cinder] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:08.727832 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b47acc-b2ad-4e4e-8e78-88fd384901d2 service cinder] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:08.732347 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05425dd-ab32-4f25-8d95-0d5c2d2f5208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:08.733220 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05425dd-ab32-4f25-8d95-0d5c2d2f5208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:08.733502 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05425dd-ab32-4f25-8d95-0d5c2d2f5208 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:08.733776 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05425dd-ab32-4f25-8d95-0d5c2d2f5208 admin admin] PATH_INFO: `/v3/users/f6b469c6eaa14a5bb3120004f2870679` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:08.734944 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05425dd-ab32-4f25-8d95-0d5c2d2f5208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6b469c6eaa14a5bb3120004f2870679)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:08.736890 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05425dd-ab32-4f25-8d95-0d5c2d2f5208 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:08.784540 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-c05425dd-ab32-4f25-8d95-0d5c2d2f5208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6b469c6eaa14a5bb3120004f2870679'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:08.796183 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-c05425dd-ab32-4f25-8d95-0d5c2d2f5208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6b469c6eaa14a5bb3120004f2870679'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:08.803424 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-c05425dd-ab32-4f25-8d95-0d5c2d2f5208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6b469c6eaa14a5bb3120004f2870679'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:08.804052 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-c05425dd-ab32-4f25-8d95-0d5c2d2f5208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6b469c6eaa14a5bb3120004f2870679'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:08.810944 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-c05425dd-ab32-4f25-8d95-0d5c2d2f5208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6b469c6eaa14a5bb3120004f2870679'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:08.816758 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1434/2883] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 12:07:08 2022] DELETE /identity/v3/users/f6b469c6eaa14a5bb3120004f2870679 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:07:08.821294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168cd86-38bd-4de2-af5c-eee3d8160655 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:08.821799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168cd86-38bd-4de2-af5c-eee3d8160655 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:08.821906 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168cd86-38bd-4de2-af5c-eee3d8160655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:08.838981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-c168cd86-38bd-4de2-af5c-eee3d8160655 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:07:08.840218 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-c168cd86-38bd-4de2-af5c-eee3d8160655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:08.897720 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1450/2884] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:07:08.908048 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05425dd-ab32-4f25-8d95-0d5c2d2f5208 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:08.909497 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-c05425dd-ab32-4f25-8d95-0d5c2d2f5208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:08.923577 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:07:08.968259 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05425dd-ab32-4f25-8d95-0d5c2d2f5208 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:08.971989 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073ff3b-eb24-43ea-afbe-a9f0adb3f5e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:08.972628 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073ff3b-eb24-43ea-afbe-a9f0adb3f5e6 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:08.972912 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073ff3b-eb24-43ea-afbe-a9f0adb3f5e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:08.973189 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073ff3b-eb24-43ea-afbe-a9f0adb3f5e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:08.982011 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073ff3b-eb24-43ea-afbe-a9f0adb3f5e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:08.983479 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073ff3b-eb24-43ea-afbe-a9f0adb3f5e6 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:08.992764 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1435/2885] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:07:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:07:09.252623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168cd86-38bd-4de2-af5c-eee3d8160655 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:09.254094 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-c168cd86-38bd-4de2-af5c-eee3d8160655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:09.268927 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168cd86-38bd-4de2-af5c-eee3d8160655 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:09.273806 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180fc73-f763-4cff-b2a2-019fec5843b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:09.274930 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180fc73-f763-4cff-b2a2-019fec5843b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:09.275278 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180fc73-f763-4cff-b2a2-019fec5843b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:09.275514 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180fc73-f763-4cff-b2a2-019fec5843b4 admin admin] PATH_INFO: `/v3/projects/da61641e14e84854994796c6edf68b47` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:09.276780 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180fc73-f763-4cff-b2a2-019fec5843b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da61641e14e84854994796c6edf68b47)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:09.278622 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180fc73-f763-4cff-b2a2-019fec5843b4 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:09.327208 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3180fc73-f763-4cff-b2a2-019fec5843b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da61641e14e84854994796c6edf68b47', 'request_id': 'req-3180fc73-f763-4cff-b2a2-019fec5843b4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:09.333470 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3180fc73-f763-4cff-b2a2-019fec5843b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da61641e14e84854994796c6edf68b47', 'request_id': 'req-3180fc73-f763-4cff-b2a2-019fec5843b4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:09.341188 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3180fc73-f763-4cff-b2a2-019fec5843b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da61641e14e84854994796c6edf68b47', 'request_id': 'req-3180fc73-f763-4cff-b2a2-019fec5843b4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:09.351666 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3180fc73-f763-4cff-b2a2-019fec5843b4 admin admin] The token cache is being invalidate because project da61641e14e84854994796c6edf68b47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 12:07:09.351959 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3180fc73-f763-4cff-b2a2-019fec5843b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:09.352612 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-3180fc73-f763-4cff-b2a2-019fec5843b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:09.354175 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1451/2886] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 12:07:09 2022] DELETE /identity/v3/projects/da61641e14e84854994796c6edf68b47 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 12:07:09.358265 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073ff3b-eb24-43ea-afbe-a9f0adb3f5e6 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:09.367843 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073ff3b-eb24-43ea-afbe-a9f0adb3f5e6 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:09.424995 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3a460-2b9f-4710-bc2f-50347ee50f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:09.425783 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3a460-2b9f-4710-bc2f-50347ee50f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:09.426066 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3a460-2b9f-4710-bc2f-50347ee50f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:09.426342 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa3a460-2b9f-4710-bc2f-50347ee50f52 admin admin] PATH_INFO: `/v3/users/0b88e3294e2141d8b28fca4a1ad73e4e` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:09.427543 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3a460-2b9f-4710-bc2f-50347ee50f52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b88e3294e2141d8b28fca4a1ad73e4e)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:09.429442 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa3a460-2b9f-4710-bc2f-50347ee50f52 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:09.473587 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-efa3a460-2b9f-4710-bc2f-50347ee50f52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b88e3294e2141d8b28fca4a1ad73e4e'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:09.486232 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-efa3a460-2b9f-4710-bc2f-50347ee50f52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b88e3294e2141d8b28fca4a1ad73e4e'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:09.492838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-efa3a460-2b9f-4710-bc2f-50347ee50f52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b88e3294e2141d8b28fca4a1ad73e4e'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:09.493735 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-efa3a460-2b9f-4710-bc2f-50347ee50f52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b88e3294e2141d8b28fca4a1ad73e4e'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:09.501068 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-efa3a460-2b9f-4710-bc2f-50347ee50f52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b88e3294e2141d8b28fca4a1ad73e4e'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:09.506840 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1436/2887] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 12:07:09 2022] DELETE /identity/v3/users/0b88e3294e2141d8b28fca4a1ad73e4e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:07:09.773177 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180fc73-f763-4cff-b2a2-019fec5843b4 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:09.775042 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-3180fc73-f763-4cff-b2a2-019fec5843b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:09.790103 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180fc73-f763-4cff-b2a2-019fec5843b4 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:09.810047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:07:09.852102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549053f2-c9ea-4307-a7b3-f7787159bdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:09.852978 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549053f2-c9ea-4307-a7b3-f7787159bdf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:09.853264 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549053f2-c9ea-4307-a7b3-f7787159bdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:09.853541 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549053f2-c9ea-4307-a7b3-f7787159bdf4 admin admin] PATH_INFO: `/v3/projects/a2da056196544268a6ab5cad08834fac` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:09.854782 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549053f2-c9ea-4307-a7b3-f7787159bdf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2da056196544268a6ab5cad08834fac)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:09.856731 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549053f2-c9ea-4307-a7b3-f7787159bdf4 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:09.908107 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-549053f2-c9ea-4307-a7b3-f7787159bdf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2da056196544268a6ab5cad08834fac', 'request_id': 'req-549053f2-c9ea-4307-a7b3-f7787159bdf4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:09.914426 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-549053f2-c9ea-4307-a7b3-f7787159bdf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2da056196544268a6ab5cad08834fac', 'request_id': 'req-549053f2-c9ea-4307-a7b3-f7787159bdf4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:09.922434 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-549053f2-c9ea-4307-a7b3-f7787159bdf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2da056196544268a6ab5cad08834fac', 'request_id': 'req-549053f2-c9ea-4307-a7b3-f7787159bdf4'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:09.945871 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-549053f2-c9ea-4307-a7b3-f7787159bdf4 admin admin] The token cache is being invalidate because project a2da056196544268a6ab5cad08834fac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 12:07:09.945871 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-549053f2-c9ea-4307-a7b3-f7787159bdf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:09.946778 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-549053f2-c9ea-4307-a7b3-f7787159bdf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:09.948714 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1452/2888] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 12:07:09 2022] DELETE /identity/v3/projects/a2da056196544268a6ab5cad08834fac => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:07:09.952666 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3a460-2b9f-4710-bc2f-50347ee50f52 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:09.954118 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-efa3a460-2b9f-4710-bc2f-50347ee50f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:09.968227 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa3a460-2b9f-4710-bc2f-50347ee50f52 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:10.000896 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:07:10.026085 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f43f33-94d2-42d1-8499-f965d0f5f99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:10.026974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f43f33-94d2-42d1-8499-f965d0f5f99e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:10.027263 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f43f33-94d2-42d1-8499-f965d0f5f99e admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:10.027590 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f43f33-94d2-42d1-8499-f965d0f5f99e admin admin] PATH_INFO: `/v3/users/0d0d4b15032741d9a82a59836cc208af` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:10.028765 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f43f33-94d2-42d1-8499-f965d0f5f99e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d0d4b15032741d9a82a59836cc208af)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:10.031197 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f43f33-94d2-42d1-8499-f965d0f5f99e admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:10.076616 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-75f43f33-94d2-42d1-8499-f965d0f5f99e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d0d4b15032741d9a82a59836cc208af'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:10.088694 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-75f43f33-94d2-42d1-8499-f965d0f5f99e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d0d4b15032741d9a82a59836cc208af'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:10.096098 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-75f43f33-94d2-42d1-8499-f965d0f5f99e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d0d4b15032741d9a82a59836cc208af'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:10.096721 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-75f43f33-94d2-42d1-8499-f965d0f5f99e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d0d4b15032741d9a82a59836cc208af'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:10.103709 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-75f43f33-94d2-42d1-8499-f965d0f5f99e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d0d4b15032741d9a82a59836cc208af'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:10.110101 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1437/2889] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 12:07:09 2022] DELETE /identity/v3/users/0d0d4b15032741d9a82a59836cc208af => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:07:10.383623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549053f2-c9ea-4307-a7b3-f7787159bdf4 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:10.385109 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-549053f2-c9ea-4307-a7b3-f7787159bdf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:10.398266 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549053f2-c9ea-4307-a7b3-f7787159bdf4 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:10.455858 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b4753-11a3-42b9-8667-7f6b62cce069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:10.456722 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b4753-11a3-42b9-8667-7f6b62cce069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:10.457006 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b4753-11a3-42b9-8667-7f6b62cce069 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:10.457280 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b4753-11a3-42b9-8667-7f6b62cce069 admin admin] PATH_INFO: `/v3/projects/6e9a6d4f5c944e80891b69156cd31790` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:10.458455 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b4753-11a3-42b9-8667-7f6b62cce069 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e9a6d4f5c944e80891b69156cd31790)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:10.460401 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b4753-11a3-42b9-8667-7f6b62cce069 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:10.505690 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-338b4753-11a3-42b9-8667-7f6b62cce069 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e9a6d4f5c944e80891b69156cd31790', 'request_id': 'req-338b4753-11a3-42b9-8667-7f6b62cce069'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:10.512234 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-338b4753-11a3-42b9-8667-7f6b62cce069 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e9a6d4f5c944e80891b69156cd31790', 'request_id': 'req-338b4753-11a3-42b9-8667-7f6b62cce069'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:10.513910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:07:10.520385 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-338b4753-11a3-42b9-8667-7f6b62cce069 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e9a6d4f5c944e80891b69156cd31790', 'request_id': 'req-338b4753-11a3-42b9-8667-7f6b62cce069'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:10.532910 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-338b4753-11a3-42b9-8667-7f6b62cce069 admin admin] The token cache is being invalidate because project 6e9a6d4f5c944e80891b69156cd31790 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 12:07:10.533223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-338b4753-11a3-42b9-8667-7f6b62cce069 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:10.533848 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-338b4753-11a3-42b9-8667-7f6b62cce069 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:10.535415 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1453/2890] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 12:07:10 2022] DELETE /identity/v3/projects/6e9a6d4f5c944e80891b69156cd31790 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:07:10.540392 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d53b8-5ece-4383-9826-e20fb0e85d80 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:10.540680 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d53b8-5ece-4383-9826-e20fb0e85d80 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:10.540961 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d53b8-5ece-4383-9826-e20fb0e85d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:10.560638 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-f68d53b8-5ece-4383-9826-e20fb0e85d80 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:07:10.562347 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-f68d53b8-5ece-4383-9826-e20fb0e85d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:10.622427 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1438/2891] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:07:10.631741 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce671ea0-0c21-499b-b4d0-713f6047119e None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:10.632151 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce671ea0-0c21-499b-b4d0-713f6047119e None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:10.632434 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce671ea0-0c21-499b-b4d0-713f6047119e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:10.651438 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-ce671ea0-0c21-499b-b4d0-713f6047119e None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:07:10.652644 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-ce671ea0-0c21-499b-b4d0-713f6047119e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:10.660577 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1454/2892] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 22 12:07:10.664733 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d53b8-5ece-4383-9826-e20fb0e85d80 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:10.674457 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d53b8-5ece-4383-9826-e20fb0e85d80 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:10.679101 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76682bd7-bd37-4bd8-9ec4-439a01742bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:10.679789 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76682bd7-bd37-4bd8-9ec4-439a01742bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:10.680076 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76682bd7-bd37-4bd8-9ec4-439a01742bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:10.680357 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76682bd7-bd37-4bd8-9ec4-439a01742bd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:10.681077 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76682bd7-bd37-4bd8-9ec4-439a01742bd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 12:07:10.681386 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76682bd7-bd37-4bd8-9ec4-439a01742bd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:10.682964 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76682bd7-bd37-4bd8-9ec4-439a01742bd0 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:10.694960 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1439/2893] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 12:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 22 12:07:10.699031 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce671ea0-0c21-499b-b4d0-713f6047119e None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:10.708945 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce671ea0-0c21-499b-b4d0-713f6047119e None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:10.713621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafeeb0-162e-4374-858f-6edba71446a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:10.714259 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafeeb0-162e-4374-858f-6edba71446a8 admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:10.714543 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafeeb0-162e-4374-858f-6edba71446a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:10.714824 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aafeeb0-162e-4374-858f-6edba71446a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:10.715539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aafeeb0-162e-4374-858f-6edba71446a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:10.717433 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aafeeb0-162e-4374-858f-6edba71446a8 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:10.744595 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1455/2894] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 12:07:10 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 12:07:10.747864 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76682bd7-bd37-4bd8-9ec4-439a01742bd0 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:10.758501 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76682bd7-bd37-4bd8-9ec4-439a01742bd0 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:10.798482 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:07:10.816948 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8b009-6f47-4a26-85db-ea06a060b7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:10.817614 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8b009-6f47-4a26-85db-ea06a060b7a2 admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:10.817901 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8b009-6f47-4a26-85db-ea06a060b7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:10.818182 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8b009-6f47-4a26-85db-ea06a060b7a2 admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:10.818860 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8b009-6f47-4a26-85db-ea06a060b7a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:10.820838 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb8b009-6f47-4a26-85db-ea06a060b7a2 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:10.840914 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1440/2895] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 12:07:10 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 12:07:10.845140 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafeeb0-162e-4374-858f-6edba71446a8 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:10.855599 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aafeeb0-162e-4374-858f-6edba71446a8 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:10.860263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fa3f8-003b-484b-b544-2955f5a70d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:10.860912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fa3f8-003b-484b-b544-2955f5a70d3e admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:10.861210 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fa3f8-003b-484b-b544-2955f5a70d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:10.861470 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fa3f8-003b-484b-b544-2955f5a70d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:10.862185 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3fa3f8-003b-484b-b544-2955f5a70d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:10.863799 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3fa3f8-003b-484b-b544-2955f5a70d3e admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:10.874582 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1456/2896] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:07:10.878330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8b009-6f47-4a26-85db-ea06a060b7a2 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:10.888279 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8b009-6f47-4a26-85db-ea06a060b7a2 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:10.893853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d66594-b64a-4229-bec2-7c80b8eb5fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:10.894546 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d66594-b64a-4229-bec2-7c80b8eb5fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:10.894831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d66594-b64a-4229-bec2-7c80b8eb5fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:10.895115 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d66594-b64a-4229-bec2-7c80b8eb5fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:10.895890 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d66594-b64a-4229-bec2-7c80b8eb5fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:10.897494 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d66594-b64a-4229-bec2-7c80b8eb5fb9 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:10.907893 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1441/2897] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:07:10.913004 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fa3f8-003b-484b-b544-2955f5a70d3e admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:10.925321 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fa3f8-003b-484b-b544-2955f5a70d3e admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:10.930375 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d50d23-b33e-4f91-9e80-4f5ce8c84c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:10.931263 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d50d23-b33e-4f91-9e80-4f5ce8c84c6d admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:10.931581 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d50d23-b33e-4f91-9e80-4f5ce8c84c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:10.931861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d50d23-b33e-4f91-9e80-4f5ce8c84c6d admin admin] PATH_INFO: `/v3/projects/a4986703a786486b84b8bc28253d2087/users/742f5db410cf44fabc0d5e2996eebcc1/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:10.947848 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d50d23-b33e-4f91-9e80-4f5ce8c84c6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4986703a786486b84b8bc28253d2087, user_id=742f5db410cf44fabc0d5e2996eebcc1, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:10.950513 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d50d23-b33e-4f91-9e80-4f5ce8c84c6d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:10.963578 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1457/2898] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 12:07:10 2022] PUT /identity/v3/projects/a4986703a786486b84b8bc28253d2087/users/742f5db410cf44fabc0d5e2996eebcc1/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:07:10.973925 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc421a-fa73-4046-a77e-db4b110d29f7 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:10.974401 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc421a-fa73-4046-a77e-db4b110d29f7 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:10.974519 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc421a-fa73-4046-a77e-db4b110d29f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:11.001535 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-47cc421a-fa73-4046-a77e-db4b110d29f7 None None] MFA Rules not processed for user `742f5db410cf44fabc0d5e2996eebcc1`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:07:11.002751 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-47cc421a-fa73-4046-a77e-db4b110d29f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:11.065225 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1442/2899] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:07:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:07:11.076269 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d50d23-b33e-4f91-9e80-4f5ce8c84c6d admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:11.077971 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-a0d50d23-b33e-4f91-9e80-4f5ce8c84c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:11.135445 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d50d23-b33e-4f91-9e80-4f5ce8c84c6d admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:11.139030 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9db1f7-2021-4dc5-a0a7-ed3d116a7aef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:11.139696 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9db1f7-2021-4dc5-a0a7-ed3d116a7aef service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:11.139981 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9db1f7-2021-4dc5-a0a7-ed3d116a7aef service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:11.140257 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9db1f7-2021-4dc5-a0a7-ed3d116a7aef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:11.148530 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9db1f7-2021-4dc5-a0a7-ed3d116a7aef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:11.150220 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9db1f7-2021-4dc5-a0a7-ed3d116a7aef service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:11.158908 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1458/2900] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:07:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:07:13.853990 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc421a-fa73-4046-a77e-db4b110d29f7 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:13.915286 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc421a-fa73-4046-a77e-db4b110d29f7 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:13.919710 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead5d0f-f648-4f5a-b916-cd1f21948145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:13.921073 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead5d0f-f648-4f5a-b916-cd1f21948145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:13.921355 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead5d0f-f648-4f5a-b916-cd1f21948145 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:13.921632 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead5d0f-f648-4f5a-b916-cd1f21948145 admin admin] PATH_INFO: `/v3/users/742f5db410cf44fabc0d5e2996eebcc1` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:13.922808 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead5d0f-f648-4f5a-b916-cd1f21948145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=742f5db410cf44fabc0d5e2996eebcc1)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:13.924739 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead5d0f-f648-4f5a-b916-cd1f21948145 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:13.974021 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-2ead5d0f-f648-4f5a-b916-cd1f21948145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '742f5db410cf44fabc0d5e2996eebcc1'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:13.978429 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:07:13.988091 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-2ead5d0f-f648-4f5a-b916-cd1f21948145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '742f5db410cf44fabc0d5e2996eebcc1'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:13.995229 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-2ead5d0f-f648-4f5a-b916-cd1f21948145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '742f5db410cf44fabc0d5e2996eebcc1'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:13.995873 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-2ead5d0f-f648-4f5a-b916-cd1f21948145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '742f5db410cf44fabc0d5e2996eebcc1'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:14.002713 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-2ead5d0f-f648-4f5a-b916-cd1f21948145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '742f5db410cf44fabc0d5e2996eebcc1'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:14.008596 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1443/2901] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 12:07:13 2022] DELETE /identity/v3/users/742f5db410cf44fabc0d5e2996eebcc1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:07:14.013081 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982d786-8c6c-4443-9118-a192887df7a0 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:14.013562 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982d786-8c6c-4443-9118-a192887df7a0 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:14.013675 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982d786-8c6c-4443-9118-a192887df7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:14.030881 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f982d786-8c6c-4443-9118-a192887df7a0 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:07:14.032093 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f982d786-8c6c-4443-9118-a192887df7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:14.088444 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1459/2902] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:07:14.098240 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead5d0f-f648-4f5a-b916-cd1f21948145 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:14.099671 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-2ead5d0f-f648-4f5a-b916-cd1f21948145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:14.155972 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead5d0f-f648-4f5a-b916-cd1f21948145 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:14.159618 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410888d-f904-42b1-802b-88f682042920 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:14.160251 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410888d-f904-42b1-802b-88f682042920 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:14.160531 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410888d-f904-42b1-802b-88f682042920 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:14.160802 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410888d-f904-42b1-802b-88f682042920 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:14.169615 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410888d-f904-42b1-802b-88f682042920 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:14.171552 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410888d-f904-42b1-802b-88f682042920 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:14.180695 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1444/2903] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:07:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:07:14.447141 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982d786-8c6c-4443-9118-a192887df7a0 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:14.448627 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f982d786-8c6c-4443-9118-a192887df7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:14.461012 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982d786-8c6c-4443-9118-a192887df7a0 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:14.465672 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e02fce-38e9-4cb0-b776-866dc1cc0d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:14.466539 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e02fce-38e9-4cb0-b776-866dc1cc0d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:14.466855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e02fce-38e9-4cb0-b776-866dc1cc0d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:14.467830 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e02fce-38e9-4cb0-b776-866dc1cc0d93 admin admin] PATH_INFO: `/v3/projects/a4986703a786486b84b8bc28253d2087` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:14.469586 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e02fce-38e9-4cb0-b776-866dc1cc0d93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4986703a786486b84b8bc28253d2087)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:14.470948 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e02fce-38e9-4cb0-b776-866dc1cc0d93 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:14.488160 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:07:14.519721 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-77e02fce-38e9-4cb0-b776-866dc1cc0d93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4986703a786486b84b8bc28253d2087', 'request_id': 'req-77e02fce-38e9-4cb0-b776-866dc1cc0d93'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:14.526071 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-77e02fce-38e9-4cb0-b776-866dc1cc0d93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4986703a786486b84b8bc28253d2087', 'request_id': 'req-77e02fce-38e9-4cb0-b776-866dc1cc0d93'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:14.533491 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-77e02fce-38e9-4cb0-b776-866dc1cc0d93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4986703a786486b84b8bc28253d2087', 'request_id': 'req-77e02fce-38e9-4cb0-b776-866dc1cc0d93'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:14.543886 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-77e02fce-38e9-4cb0-b776-866dc1cc0d93 admin admin] The token cache is being invalidate because project a4986703a786486b84b8bc28253d2087 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 12:07:14.544176 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-77e02fce-38e9-4cb0-b776-866dc1cc0d93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:14.544793 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-77e02fce-38e9-4cb0-b776-866dc1cc0d93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:07:14.546325 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1460/2904] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 12:07:14 2022] DELETE /identity/v3/projects/a4986703a786486b84b8bc28253d2087 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:07:14.551212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244ca4f-a0c2-4317-b3af-0ccd7327c528 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:14.551524 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244ca4f-a0c2-4317-b3af-0ccd7327c528 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:14.551807 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244ca4f-a0c2-4317-b3af-0ccd7327c528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:14.569426 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-a244ca4f-a0c2-4317-b3af-0ccd7327c528 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:07:14.570623 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-a244ca4f-a0c2-4317-b3af-0ccd7327c528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:14.630882 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1445/2905] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:07:14.638744 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c2b34-36a0-409a-ac67-7904e1b9bcc2 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:14.639575 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c2b34-36a0-409a-ac67-7904e1b9bcc2 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:14.639866 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c2b34-36a0-409a-ac67-7904e1b9bcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:14.658567 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-4f8c2b34-36a0-409a-ac67-7904e1b9bcc2 None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:07:14.659822 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-4f8c2b34-36a0-409a-ac67-7904e1b9bcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:14.667747 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1461/2906] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:07:14.671795 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244ca4f-a0c2-4317-b3af-0ccd7327c528 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:14.681304 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244ca4f-a0c2-4317-b3af-0ccd7327c528 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:14.686522 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1681a2-7c17-4d87-aadc-d83900221ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:14.687179 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1681a2-7c17-4d87-aadc-d83900221ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:14.687503 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1681a2-7c17-4d87-aadc-d83900221ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:14.687809 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1681a2-7c17-4d87-aadc-d83900221ad2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:14.688531 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1681a2-7c17-4d87-aadc-d83900221ad2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86768) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 22 12:07:14.688839 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1681a2-7c17-4d87-aadc-d83900221ad2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:14.690481 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1681a2-7c17-4d87-aadc-d83900221ad2 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:14.702335 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1446/2907] 10.222.0.36 () {58 vars in 1152 bytes} [Wed Jun 22 12:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 22 12:07:14.706519 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c2b34-36a0-409a-ac67-7904e1b9bcc2 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:14.715693 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c2b34-36a0-409a-ac67-7904e1b9bcc2 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:14.720345 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d625f9-b9c1-45cc-9c21-ca7c11a4c03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:14.720983 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d625f9-b9c1-45cc-9c21-ca7c11a4c03d admin admin] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:14.721265 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d625f9-b9c1-45cc-9c21-ca7c11a4c03d admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:14.721541 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d625f9-b9c1-45cc-9c21-ca7c11a4c03d admin admin] PATH_INFO: `/v3/projects` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:14.722228 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d625f9-b9c1-45cc-9c21-ca7c11a4c03d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:14.724160 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d625f9-b9c1-45cc-9c21-ca7c11a4c03d admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:14.749662 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1462/2908] 10.222.0.36 () {60 vars in 1152 bytes} [Wed Jun 22 12:07:14 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 12:07:14.753640 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1681a2-7c17-4d87-aadc-d83900221ad2 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:14.763450 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1681a2-7c17-4d87-aadc-d83900221ad2 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:14.820270 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82b3c4-156d-44d2-b4de-7f5b5b427c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:14.820931 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82b3c4-156d-44d2-b4de-7f5b5b427c0a admin admin] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:14.821214 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82b3c4-156d-44d2-b4de-7f5b5b427c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:14.821495 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82b3c4-156d-44d2-b4de-7f5b5b427c0a admin admin] PATH_INFO: `/v3/users` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:14.822167 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82b3c4-156d-44d2-b4de-7f5b5b427c0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:14.824088 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82b3c4-156d-44d2-b4de-7f5b5b427c0a admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:14.831222 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:07:14.846692 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1447/2909] 10.222.0.36 () {60 vars in 1143 bytes} [Wed Jun 22 12:07:14 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 22 12:07:14.850621 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d625f9-b9c1-45cc-9c21-ca7c11a4c03d admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:14.861473 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d625f9-b9c1-45cc-9c21-ca7c11a4c03d admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:14.866038 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fcd70-065f-4f58-8359-18e1108204f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:14.866686 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fcd70-065f-4f58-8359-18e1108204f2 admin admin] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:14.866967 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fcd70-065f-4f58-8359-18e1108204f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:14.867243 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fcd70-065f-4f58-8359-18e1108204f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:14.868003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545fcd70-065f-4f58-8359-18e1108204f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:14.869576 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545fcd70-065f-4f58-8359-18e1108204f2 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:14.879757 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1463/2910] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:07:14.883418 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82b3c4-156d-44d2-b4de-7f5b5b427c0a admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:14.893390 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82b3c4-156d-44d2-b4de-7f5b5b427c0a admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:14.898382 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb7e00-6eb9-46b8-8064-ab149eaffe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:14.899296 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb7e00-6eb9-46b8-8064-ab149eaffe8e admin admin] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:14.899630 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb7e00-6eb9-46b8-8064-ab149eaffe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:14.899889 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb7e00-6eb9-46b8-8064-ab149eaffe8e admin admin] PATH_INFO: `/v3/roles` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:14.900613 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb7e00-6eb9-46b8-8064-ab149eaffe8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:14.902184 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb7e00-6eb9-46b8-8064-ab149eaffe8e admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:14.912930 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1448/2911] 10.222.0.36 () {58 vars in 1121 bytes} [Wed Jun 22 12:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 22 12:07:14.917047 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fcd70-065f-4f58-8359-18e1108204f2 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:14.929000 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fcd70-065f-4f58-8359-18e1108204f2 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:14.933647 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745f0a6-213a-4311-b7cb-a5fb34ed57a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:14.934523 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745f0a6-213a-4311-b7cb-a5fb34ed57a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:14.934808 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745f0a6-213a-4311-b7cb-a5fb34ed57a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:14.935087 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745f0a6-213a-4311-b7cb-a5fb34ed57a2 admin admin] PATH_INFO: `/v3/projects/e10970e971fb4b59bc2a5b816a2b376b/users/8138ccacb8e24c7f8c68d954d54baad7/roles/f2336919c5104f6a8210e99994f4d455` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:14.949765 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745f0a6-213a-4311-b7cb-a5fb34ed57a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e10970e971fb4b59bc2a5b816a2b376b, user_id=8138ccacb8e24c7f8c68d954d54baad7, role_id=f2336919c5104f6a8210e99994f4d455)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:14.952453 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745f0a6-213a-4311-b7cb-a5fb34ed57a2 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:14.964861 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1464/2912] 10.222.0.36 () {60 vars in 1482 bytes} [Wed Jun 22 12:07:14 2022] PUT /identity/v3/projects/e10970e971fb4b59bc2a5b816a2b376b/users/8138ccacb8e24c7f8c68d954d54baad7/roles/f2336919c5104f6a8210e99994f4d455 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 22 12:07:14.970363 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46401c-643a-4c72-8c32-9dbe01096aab None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:14.970693 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46401c-643a-4c72-8c32-9dbe01096aab None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:14.971037 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46401c-643a-4c72-8c32-9dbe01096aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:14.989121 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-cd46401c-643a-4c72-8c32-9dbe01096aab None None] MFA Rules not processed for user `8441761690594b9d8a423147d5d6d62d`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:07:14.990320 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-cd46401c-643a-4c72-8c32-9dbe01096aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:15.047957 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1449/2913] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:07:15.057300 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745f0a6-213a-4311-b7cb-a5fb34ed57a2 admin admin] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:15.059068 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-6745f0a6-213a-4311-b7cb-a5fb34ed57a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:15.114660 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745f0a6-213a-4311-b7cb-a5fb34ed57a2 admin admin] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:15.118318 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e028a2-846a-4dc1-a723-bcbe1761f7ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:15.118952 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e028a2-846a-4dc1-a723-bcbe1761f7ab service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:15.119237 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e028a2-846a-4dc1-a723-bcbe1761f7ab service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:15.119544 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e028a2-846a-4dc1-a723-bcbe1761f7ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:15.128509 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e028a2-846a-4dc1-a723-bcbe1761f7ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:15.129935 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e028a2-846a-4dc1-a723-bcbe1761f7ab service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:15.139832 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1465/2914] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:07:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:07:15.726172 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044542a-d592-40dc-90a0-c401caa62ddf None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:15.726467 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044542a-d592-40dc-90a0-c401caa62ddf None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:15.726750 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044542a-d592-40dc-90a0-c401caa62ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:15.752815 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-8044542a-d592-40dc-90a0-c401caa62ddf None None] MFA Rules not processed for user `8138ccacb8e24c7f8c68d954d54baad7`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:07:15.754033 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-8044542a-d592-40dc-90a0-c401caa62ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:15.824062 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1450/2915] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:07:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:07:15.834831 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e028a2-846a-4dc1-a723-bcbe1761f7ab service neutron] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:15.836271 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-72e028a2-846a-4dc1-a723-bcbe1761f7ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:15.893335 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:07:15.900540 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e028a2-846a-4dc1-a723-bcbe1761f7ab service neutron] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:15.905825 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dcf155-7813-42cb-b823-cd45048007fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ceb170ac6353495b9d1dc6723d59567a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:15.906546 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcf155-7813-42cb-b823-cd45048007fa service cinder] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:15.906829 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcf155-7813-42cb-b823-cd45048007fa service cinder] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:15.907102 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcf155-7813-42cb-b823-cd45048007fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:15.916409 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dcf155-7813-42cb-b823-cd45048007fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:15.917855 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dcf155-7813-42cb-b823-cd45048007fa service cinder] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:15.929810 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1466/2916] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:07:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:07:17.265927 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad352652-973d-4ae3-9293-0ab526be8f05 None None] REQUEST_METHOD: `POST` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:17.266421 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad352652-973d-4ae3-9293-0ab526be8f05 None None] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:17.266527 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad352652-973d-4ae3-9293-0ab526be8f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:17.285766 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.auth.core [None req-ad352652-973d-4ae3-9293-0ab526be8f05 None None] MFA Rules not processed for user `8138ccacb8e24c7f8c68d954d54baad7`. Rule list: `[]` (Enabled: `True`). {{(pid=86768) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:07:17.286974 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ad352652-973d-4ae3-9293-0ab526be8f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:17.294192 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1451/2917] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:07:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:07:19.458764 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dcf155-7813-42cb-b823-cd45048007fa service cinder] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:07:19.460268 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-e5dcf155-7813-42cb-b823-cd45048007fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:19.472788 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dcf155-7813-42cb-b823-cd45048007fa service cinder] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:07:19.476371 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68073cd9-ee42-4f23-85f7-d6c4b983f12b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:07:19.477003 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68073cd9-ee42-4f23-85f7-d6c4b983f12b service neutron] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:07:19.477290 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68073cd9-ee42-4f23-85f7-d6c4b983f12b service neutron] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:07:19.477568 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68073cd9-ee42-4f23-85f7-d6c4b983f12b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:07:19.479307 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-68073cd9-ee42-4f23-85f7-d6c4b983f12b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:07:19.489223 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68073cd9-ee42-4f23-85f7-d6c4b983f12b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:07:19.490638 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68073cd9-ee42-4f23-85f7-d6c4b983f12b service neutron] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:07:19.498562 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1467/2918] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:07:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:07:27.279846 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:07:29.495082 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=12 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:08:30.654099 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad352652-973d-4ae3-9293-0ab526be8f05 None None] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:08:30.655556 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-ad352652-973d-4ae3-9293-0ab526be8f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:08:30.793132 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad352652-973d-4ae3-9293-0ab526be8f05 None None] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:08:30.798162 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5868a-cfcf-48db-98e3-a1dfd1ff1061 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93630ed5e6dc4a5f958bc7cf1a68a26c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:08:30.798800 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5868a-cfcf-48db-98e3-a1dfd1ff1061 service placement] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:08:30.799080 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5868a-cfcf-48db-98e3-a1dfd1ff1061 service placement] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:08:30.799359 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5868a-cfcf-48db-98e3-a1dfd1ff1061 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:08:30.801126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-97b5868a-cfcf-48db-98e3-a1dfd1ff1061 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:08:30.812810 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b5868a-cfcf-48db-98e3-a1dfd1ff1061 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:08:30.814370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b5868a-cfcf-48db-98e3-a1dfd1ff1061 service placement] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:08:30.824126 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1452/2919] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:08:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:08:31.213957 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6387d14-ac35-44ea-95ec-ddcd3b2d0965 None None] REQUEST_METHOD: `POST` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:08:31.215085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6387d14-ac35-44ea-95ec-ddcd3b2d0965 None None] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:08:31.215085 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6387d14-ac35-44ea-95ec-ddcd3b2d0965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:08:31.231956 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.auth.core [None req-f6387d14-ac35-44ea-95ec-ddcd3b2d0965 None None] MFA Rules not processed for user `8138ccacb8e24c7f8c68d954d54baad7`. Rule list: `[]` (Enabled: `True`). {{(pid=86769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 22 12:08:31.233155 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f6387d14-ac35-44ea-95ec-ddcd3b2d0965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:08:31.240253 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1468/2920] 10.222.0.36 () {58 vars in 957 bytes} [Wed Jun 22 12:08:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 22 12:08:31.478212 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5868a-cfcf-48db-98e3-a1dfd1ff1061 service placement] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:08:31.496380 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5868a-cfcf-48db-98e3-a1dfd1ff1061 service placement] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:08:31.499949 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a93051-9028-41b7-bbe6-1dff409eae10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c00d0bb777a044a3a4d4540d187d938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:08:31.500576 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a93051-9028-41b7-bbe6-1dff409eae10 service neutron] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:08:31.500855 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a93051-9028-41b7-bbe6-1dff409eae10 service neutron] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:08:31.501125 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a93051-9028-41b7-bbe6-1dff409eae10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:08:31.502853 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b1a93051-9028-41b7-bbe6-1dff409eae10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:08:31.648083 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a93051-9028-41b7-bbe6-1dff409eae10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:08:31.649778 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a93051-9028-41b7-bbe6-1dff409eae10 service neutron] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:08:31.658403 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1453/2921] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:08:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:08:31.976423 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6387d14-ac35-44ea-95ec-ddcd3b2d0965 None None] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:08:31.977874 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-f6387d14-ac35-44ea-95ec-ddcd3b2d0965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:08:31.993199 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6387d14-ac35-44ea-95ec-ddcd3b2d0965 None None] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:08:31.998757 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17df7f-dc47-4f11-afa9-33e66b59e3b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93630ed5e6dc4a5f958bc7cf1a68a26c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:08:31.999427 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17df7f-dc47-4f11-afa9-33e66b59e3b8 service placement] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:08:31.999711 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17df7f-dc47-4f11-afa9-33e66b59e3b8 service placement] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:08:31.999989 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17df7f-dc47-4f11-afa9-33e66b59e3b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:08:32.001752 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-cb17df7f-dc47-4f11-afa9-33e66b59e3b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:08:32.013615 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17df7f-dc47-4f11-afa9-33e66b59e3b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:08:32.015247 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17df7f-dc47-4f11-afa9-33e66b59e3b8 service placement] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:08:32.029241 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1469/2922] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:08:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:08:41.654312 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=78 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:08:42.024719 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=9 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:08:42.234202 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a93051-9028-41b7-bbe6-1dff409eae10 service neutron] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:08:42.235945 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-b1a93051-9028-41b7-bbe6-1dff409eae10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:08:42.323573 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a93051-9028-41b7-bbe6-1dff409eae10 service neutron] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:08:42.329262 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6bf47-411a-4ce0-98e1-65a23fb593e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '479a376dcac14c509c1c55f0046d88cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:08:42.329908 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6bf47-411a-4ce0-98e1-65a23fb593e0 service nova] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:08:42.330219 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6bf47-411a-4ce0-98e1-65a23fb593e0 service nova] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:08:42.330491 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6bf47-411a-4ce0-98e1-65a23fb593e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:08:42.332622 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-cac6bf47-411a-4ce0-98e1-65a23fb593e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:08:42.345866 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6bf47-411a-4ce0-98e1-65a23fb593e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:08:42.347476 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6bf47-411a-4ce0-98e1-65a23fb593e0 service nova] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:08:42.359657 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1454/2923] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:08:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:08:52.354022 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=37 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:09:35.764544 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17df7f-dc47-4f11-afa9-33e66b59e3b8 service placement] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:09:35.775979 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17df7f-dc47-4f11-afa9-33e66b59e3b8 service placement] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:09:35.780912 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b8dcb-8cec-4b7f-adc4-1e7749e94158 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93630ed5e6dc4a5f958bc7cf1a68a26c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:09:35.781549 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b8dcb-8cec-4b7f-adc4-1e7749e94158 service placement] REQUEST_METHOD: `GET` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:09:35.781866 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b8dcb-8cec-4b7f-adc4-1e7749e94158 service placement] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:09:35.782695 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b8dcb-8cec-4b7f-adc4-1e7749e94158 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:09:35.784484 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.fernet_utils [None req-8c2b8dcb-8cec-4b7f-adc4-1e7749e94158 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:09:35.795946 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b8dcb-8cec-4b7f-adc4-1e7749e94158 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:09:35.797523 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b8dcb-8cec-4b7f-adc4-1e7749e94158 service placement] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:09:35.807442 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1470/2924] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:09:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:09:35.827953 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6bf47-411a-4ce0-98e1-65a23fb593e0 service nova] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:09:35.859139 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6bf47-411a-4ce0-98e1-65a23fb593e0 service nova] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:09:35.863936 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58a3fa-9759-4b9c-bfe1-31460b0e5c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:09:35.864788 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58a3fa-9759-4b9c-bfe1-31460b0e5c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:09:35.865064 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58a3fa-9759-4b9c-bfe1-31460b0e5c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:09:35.865338 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58a3fa-9759-4b9c-bfe1-31460b0e5c66 admin admin] PATH_INFO: `/v3/users/8138ccacb8e24c7f8c68d954d54baad7` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:09:35.866488 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58a3fa-9759-4b9c-bfe1-31460b0e5c66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8138ccacb8e24c7f8c68d954d54baad7)` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:09:35.868422 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58a3fa-9759-4b9c-bfe1-31460b0e5c66 admin admin] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:09:35.913330 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-5c58a3fa-9759-4b9c-bfe1-31460b0e5c66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8138ccacb8e24c7f8c68d954d54baad7'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:09:35.925413 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-5c58a3fa-9759-4b9c-bfe1-31460b0e5c66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8138ccacb8e24c7f8c68d954d54baad7'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:09:35.932183 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-5c58a3fa-9759-4b9c-bfe1-31460b0e5c66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8138ccacb8e24c7f8c68d954d54baad7'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:09:35.933185 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-5c58a3fa-9759-4b9c-bfe1-31460b0e5c66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8138ccacb8e24c7f8c68d954d54baad7'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:09:35.941088 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.notifications [None req-5c58a3fa-9759-4b9c-bfe1-31460b0e5c66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8138ccacb8e24c7f8c68d954d54baad7'} {{(pid=86768) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:09:35.947110 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1455/2925] 10.222.0.36 () {58 vars in 1223 bytes} [Wed Jun 22 12:09:35 2022] DELETE /identity/v3/users/8138ccacb8e24c7f8c68d954d54baad7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:09:36.112054 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b8dcb-8cec-4b7f-adc4-1e7749e94158 service placement] Authenticating user token {{(pid=86769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:09:36.128294 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b8dcb-8cec-4b7f-adc4-1e7749e94158 service placement] Validating token access rules against request {{(pid=86769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:09:36.184200 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeac6f9-d7ae-4ccc-8f69-57bf5c8c4033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8441761690594b9d8a423147d5d6d62d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77776cd83344a69a0a4fbb7efadd00f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:09:36.185056 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeac6f9-d7ae-4ccc-8f69-57bf5c8c4033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:09:36.185345 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeac6f9-d7ae-4ccc-8f69-57bf5c8c4033 admin admin] SCRIPT_NAME: `/identity` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:09:36.185619 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeac6f9-d7ae-4ccc-8f69-57bf5c8c4033 admin admin] PATH_INFO: `/v3/projects/e10970e971fb4b59bc2a5b816a2b376b` {{(pid=86769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:09:36.186792 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeac6f9-d7ae-4ccc-8f69-57bf5c8c4033 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e10970e971fb4b59bc2a5b816a2b376b)` {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:09:36.189295 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeac6f9-d7ae-4ccc-8f69-57bf5c8c4033 admin admin] RBAC: Authorization granted {{(pid=86769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:09:36.233634 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-dfeac6f9-d7ae-4ccc-8f69-57bf5c8c4033 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e10970e971fb4b59bc2a5b816a2b376b', 'request_id': 'req-dfeac6f9-d7ae-4ccc-8f69-57bf5c8c4033'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:09:36.239562 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-dfeac6f9-d7ae-4ccc-8f69-57bf5c8c4033 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e10970e971fb4b59bc2a5b816a2b376b', 'request_id': 'req-dfeac6f9-d7ae-4ccc-8f69-57bf5c8c4033'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:09:36.246537 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-dfeac6f9-d7ae-4ccc-8f69-57bf5c8c4033 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e10970e971fb4b59bc2a5b816a2b376b', 'request_id': 'req-dfeac6f9-d7ae-4ccc-8f69-57bf5c8c4033'} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:09:36.269707 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-dfeac6f9-d7ae-4ccc-8f69-57bf5c8c4033 admin admin] The token cache is being invalidate because project e10970e971fb4b59bc2a5b816a2b376b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 22 12:09:36.270002 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-dfeac6f9-d7ae-4ccc-8f69-57bf5c8c4033 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:09:36.270623 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG keystone.notifications [None req-dfeac6f9-d7ae-4ccc-8f69-57bf5c8c4033 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 22 12:09:36.272183 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: [pid: 86769|app: 0|req: 1471/2926] 10.222.0.36 () {58 vars in 1232 bytes} [Wed Jun 22 12:09:36 2022] DELETE /identity/v3/projects/e10970e971fb4b59bc2a5b816a2b376b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 22 12:09:45.943575 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:09:46.267484 dv-cinder-836973-9-iscsi devstack@keystone.service[86769]: DEBUG dbcounter [-] [86769] Writing DB stats keystone:SELECT=62,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 22 12:13:36.472244 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58a3fa-9759-4b9c-bfe1-31460b0e5c66 admin admin] Authenticating user token {{(pid=86768) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 22 12:13:36.473651 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-5c58a3fa-9759-4b9c-bfe1-31460b0e5c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:13:36.541273 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58a3fa-9759-4b9c-bfe1-31460b0e5c66 admin admin] Validating token access rules against request {{(pid=86768) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 22 12:13:36.597887 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9c82d-847b-4140-9a4a-758fcd830bd1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93630ed5e6dc4a5f958bc7cf1a68a26c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '47d2bb094c9843c9a7726f1d672d2433', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86768) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 22 12:13:36.598538 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9c82d-847b-4140-9a4a-758fcd830bd1 service placement] REQUEST_METHOD: `GET` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 22 12:13:36.598831 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9c82d-847b-4140-9a4a-758fcd830bd1 service placement] SCRIPT_NAME: `/identity` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 22 12:13:36.599102 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9c82d-847b-4140-9a4a-758fcd830bd1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86768) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 22 12:13:36.600854 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.fernet_utils [None req-48d9c82d-847b-4140-9a4a-758fcd830bd1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86768) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 22 12:13:36.611849 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d9c82d-847b-4140-9a4a-758fcd830bd1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 22 12:13:36.613391 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d9c82d-847b-4140-9a4a-758fcd830bd1 service placement] RBAC: Authorization granted {{(pid=86768) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 22 12:13:36.625370 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: [pid: 86768|app: 0|req: 1456/2927] 10.222.0.36 () {60 vars in 1365 bytes} [Wed Jun 22 12:13:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 22 12:13:46.620867 dv-cinder-836973-9-iscsi devstack@keystone.service[86768]: DEBUG dbcounter [-] [86768] Writing DB stats keystone:SELECT=58 {{(pid=86768) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}